首页|基于稀疏正则双层优化的个性化联邦学习

基于稀疏正则双层优化的个性化联邦学习

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
个性化联邦学习侧重于为各客户端提供个性化模型,旨在提高对异构数据的处理性能,然而现有的个性化联邦学习算法大多以增加客户端参数量为代价提高个性化模型的性能,使计算变得复杂.为了解决此问题,文中提出基于稀疏正则双层优化的个性化联邦学习算法(Personalized Federated Learning Based on Sparsity Regularized Bi-level Optimization,pFedSRB),在客户端的个性化更新中引入l1 范数稀疏正则化,提升个性化模型的稀疏度,避免不必要的客户端参数更新,降低模型复杂度.将个性化联邦学习建模为双层优化问题,内层优化采用交替方向乘子法,可提高学习速度.在4个联邦学习基准数据集上的实验表明,pFedSRB在异构数据上表现出色,在提高模型性能的同时有效降低训练用时和空间成本.
Personalized Federated Learning Based on Sparsity Regularized Bi-level Optimization
Personalized federated learning focuses on providing personalized model for each client,aiming to improve the processing performance on statistically heterogeneous data.However,most existing personalized federated learning algorithms enhance the performance of personalized models at the cost of increasing the number of client parameters and making computation more complex.To address this issue,a personalized federated learning algorithm based on sparsity regularized bi-level optimization(pFedSRB)is proposed in this paper.The l1 norm sparse regularization is introduced into the personalized update of each client to enhance the sparsity of the personalized model,avoid unnecessary parameter updates of clients,and reduce model complexity.The personalized federated learning problem is formulated as a bi-level optimization problem,and the inner-level optimization of pFedSRB is solved by the alternating direction method of multipliers to improve the learning speed.Experiments on four federated learning benchmark datasets demonstrate that pFedSRB performs well on heterogeneous data,effectively improving model performance while reducing the time and memory costs required for training.

Personalized Federated LearningSparse RegularizationNon-Independently and Identical-ly Distributed(Non-IID)Alternating Direction Method of Multipliers(ADMM)

刘希、刘博、季繁繁、袁晓彤

展开 >

南京信息工程大学 计算机学院 南京 210044

Walmart Global Tech Hub,Sunnyvale,CA 94086,USA

南京信息工程大学 电子与信息工程学院 南京 210044

南京大学 计算机软件新技术全国重点实验室 南京 210023

南京大学 智能科学与技术学院 苏州 215163

展开 >

个性化联邦学习 稀疏正则化 非独立同分布(Non-IID) 交替方向乘子法(ADMM)

国家自然科学基金项目国家自然科学基金项目科技创新2030-"新一代人工智能"重大项目

U21B2049619360052018AAA0100400

2024

10.16451/j.cnki.issn1003-6059.202405006

模式识别与人工智能
中国自动化学会,国家智能计算机研究开发中心,中国科学院合肥智能机械研究所

模式识别与人工智能

CSTPCD北大核心
影响因子：0.954
ISSN：1003-6059
年,卷(期)：2024.37(5)
  • 1