Review of Evolution of Large Language Model Algorithms
The large language model based on the Transformer architecture shows powerful capabilities,and it is a major progress towards artifi-cial general intelligence(AGI).The evolution of large language model architecture and algorithms is divided into two technical paths:improving the inference efficiency and model capability.The mainstream technical solutions and ideas for the two technical routes are described.Meth-ods for improving inference efficiency include distributed inference,computing optimization,memory access optimization,and quantification.To improve model capabilities,new architectures such as mixture of experts(MoE)and state space model(SSM)are introduced.
NETL
NSTL
万方数据
