Survey on Transmission Optimization Technologies for Federated Large Language Model Training
With the rapid development of artificial intelligence technology,various types of large language models are emerging.However,most users and datasets participating in dedicated large language models have certain requirements for privacy and se-curity,the data security and privacy issues need to be solved urgently,and federated large language models have emerged and gained more and more attention.Due to the huge data volume of large language models and the distributed architecture of federa-ted learning,a large number of model exchanges between a large number of participating nodes and cloud servers result in high communication costs.In order to improve the model convergence rate,researchers have investigated transmission optimization techniques for federated large language model training.This paper analyzes the challenges of federated large language models,re-views the optimization problems of transmission optimization methods based on model fine-tuning,transmission optimization methods based on model structure compression,and transmission optimization based on distributed parallel processing;introduces existing open-source federated large language models and the transmission optimization techniques used,and gives an outlook on future research directions.
Federated learningLarge language modelsTransmission optimizationCommunication overheadModel compression
万方数据
维普
