首页|Gradient sparsification for efficient wireless federated learning with differential privacy

Gradient sparsification for efficient wireless federated learning with differential privacy

扫码查看
原文链接
  • 万方数据
  • 维普
Federated learning(FL)enables distributed clients to collaboratively train a machine learning model without sharing raw data with each other.However,it suffers from the leakage of private informa-tion from uploading models.In addition,as the model size grows,the training latency increases due to the limited transmission bandwidth and model performance degradation while using differential privacy(DP)protection.In this paper,we propose a gradient sparsification empowered FL framework with DP over wire-less channels,to improve training efficiency without sacrificing convergence performance.Specifically,we first design a random sparsification algorithm to retain a fraction of the gradient elements in each client's local model,thereby mitigating the performance degradation induced by DP and reducing the number of transmission parameters over wireless channels.Then,we analyze the convergence bound of the proposed algorithm,by modeling a non-convex FL problem.Next,we formulate a time-sequential stochastic optimiza-tion problem for minimizing the developed convergence bound,under the constraints of transmit power,the average transmitting delay,as well as the client's DP requirement.Utilizing the Lyapunov drift-plus-penalty framework,we develop an analytical solution to the optimization problem.Extensive experiments have been implemented on three real-life datasets to demonstrate the effectiveness of our proposed algorithm.We show that our proposed algorithms can fully exploit the interworking between communication and computation to outperform the baselines,i.e.,random scheduling,round robin,and delay-minimization algorithms.

federated learningdifferential privacygradient sparsificationLyapunov driftconvergence analysis

Kang WEI、Jun LI、Chuan MA、Ming DING、Feng SHU、Haitao ZHAO、Wen CHEN、Hongbo ZHU

展开 >

School of Electronic and Optical Engineering,Nanjing University of Science and Technology,Nanjing 210096,China

Zhejiang Lab,Hangzhou 311121,China

Key Laboratory of Computer Network and Information Integration(Southeast University),Ministry of Education,Nanjing 211189,China

Data61,Commonwealth Scientific and Industrial Research Organisation,Sydney 2015,Australia

School of Information and Communication Engineering,Hainan University,Haikou 570228,China

School of Communications and Information Engineering,Nanjing University of Posts and Telecommunications,Nanjing 210003,China

School of Electronic Information and Electrical Engineering,Shanghai Jiao Tong University,Shanghai 200240,China

展开 >

国家自然科学基金国家自然科学基金国家自然科学基金国家自然科学基金国家重点研发计划中央高校基本科研业务费专项Key Technologies Research and Development Program of Jiangsu(Prospective and Key Technologies for Industry)Key Technologies Research and Development Program of Jiangsu(Prospective and Key Technologies for Industry)Future Network Grant of Provincial Education Board in JiangsuMajor Science and Technology Plan of Hainan ProvinceScientific Research Fund Project of Hainan UniversityYouth Foundation Project of Zhejiang LabCollaborative Innovation Center of Information Technology,Hainan UniversitySciences and Technology Commission of Shanghai MunicipalitySciences and Technology Commission of Shanghai MunicipalitySciences and Technology Commission of Shanghai Municipality

620712966200217062071234U22A20022020YFB180770030921013104BE2023022BE2023022-2ZDKJ2021022KYQDZR-21008K2023PD0AA01XTCX2022XXC0722JC140400020JC1416502PKX2021-D02

2024

10.1007/s11432-023-3918-9

中国科学:信息科学(英文版)
中国科学院

中国科学:信息科学(英文版)

CSTPCDEI
影响因子：0.715
ISSN：1674-733X
年,卷(期)：2024.67(4)
  • 38