Large model computing infrastructure technological trends,key challenges,and development trajectories
Starting from the latest technological development trends of large models, this paper first analyzes the architectural characteristics and computing power demand features of multimodal, long sequence, and mixture of experts models. Further, it focuses on the requirements of the latest large models for massive computing power scale and complex communication patterns. It quantitatively analyzes the current development problems and technical challenges faced by large model computing infrastructure from two aspects: computating efficiency and cluster interconnection technology. Finally, it proposes a high-quality computing infrastructure development trajectory oriented by applications, centered on systems, and targeted at efficiency.
multimodal modellong sequence modelmixture of experts modelcomputating efficiencycluster interconnectionhigh-quality computing power
国家科技期刊平台
NSTL
万方数据
