The major advances in key technologies and applications of large models since the release of ChatGPT are presented.In terms of large model design,the model scale is increasing,but it has slowed down.Longer context and multi-mode have become the mainstream,and the computational efficiency has been significantly improved.In terms of model training,the focus has shifted from simply seeking a larger quantity of data to a more focused approach on the diversity and quality of data,especially how to train large models using synthetic data.This is an essential direction towards achieving artificial general intelligence(AGI).In terms of model inference,model quantification and inference engine optimization greatly reduce the cost of model use,and emerging algorithms such as speculative sampling gradually ma-ture.At the application level,Agent technology has made significant progress,playing a critical role in addressing the inherent limitations of large models.More and more enterprises are beginning to plan,develop,and utilize large models,and the enterprise-level large model appli-cation architecture is becoming increasingly mature,focusing on scenarios,technologies,and algorithms to accelerate the closing loop of large model commercial value.
large modelmodel traininginference acceleratinglarge model safetyAgent
NETL
NSTL
万方数据
