面向半异步协同场景的联邦学习数据共享方法
Data sharing for semi-asynchronous federated learning
胡智尧 1于淼 1田开元1
作者信息
摘要
在半异步协同场景中,各单位采用联邦学习数据共享方法协同训练模型时存在频率不均衡的问题,原因是各单位的响应时间存在较大差异.响应时间较短的单位能以较快的训练频率更新模型,故半异步协同场景下难以从训练频率较慢的单位学习知识,导致其性能下降.针对此问题,提出了一种可感知响应时间分布的联邦学习算法,通过对参与训练的各单位进行分组,再利用有向非循环图对各个小组进行调度,以并行或串行的方式执行组内迭代训练,实现各单位训练频率均衡化.实验结果表明:该方法较传统的半异步联邦学习数据共享方法,在训练效率和模型预测性能上均有明显改善.
Abstract
In the semi-asynchronous scenario,there is a problem of unbalanced frequency when each unit uses the federated learning data-sharing method for training because the response time of each unit is quite different.Units with less response time can update the model with faster training frequen-cy,so in this case it is difficult to effectively learn knowledge from units with less response time,which affects its performance.To solve this problem,a federated learning algorithm that senses the distribution of response time was proposed.By grouping the units participating in the training,a di-rected acyclic graph was used to guide each training group to perform iterative training within the group in a parallel or serial manner,so as to achieve a balanced training frequency.Experimental re-sults show that the above mentioned method significantly improves the training efficiency and model prediction compared with the traditional semi-asynchronous federated learning data-sharing method.
关键词
联邦学习/训练频率/半异步/响应时间/有向非循环图Key words
federated learning/training frequency/semi-asynchronism/response time/directed acyclic graph引用本文复制引用
出版年
2024
NETL
NSTL
万方数据