联邦学习(federated learning，FL)是一种以保护客户隐私数据为中心的分布式处理网络，为解决隐私泄露问题提供了前景良好的解决方案。然而，FL的一个主要困境是高度非独立同分布(non-independent and identically distributed，non-ⅡD)的数据会导致全局模型性能很差。尽管相关研究已经探讨了这个问题，但本文发现当面对non-ⅡD数据、不稳定的客户端参与以及深度模型时，现有方案和标准基线FedAvg相比，只有微弱的优势或甚至更差，因此严重阻碍了 FL的隐私保护应用价值。为解决这个问题，本文提出了 一种对non-ⅡD数据鲁棒的优化方案:FedUp。该方案在保留FL隐私保护特点的前提下，进一步提升了全局模型的泛化鲁棒性。FedUp的核心思路是最小化全局经验损失函数的上限来保证模型具有低的泛化误差。大量仿真实验表明，FedUp 显著优于现有方案，并对高度non-ⅡD数据以及不稳定和大规模客户端的参与具有鲁棒性。
Enhancing generalization robustness of federated learning in highly heterogeneous environments
Federated learning(FL)is a distributed processing network that focuses on protecting client privacy data,providing a promising solution for addressing privacy leakage issues.However,a major quagmire in FL is to train clients'models over significantly non-independent and identically distributed(non-ⅡD)data,which would lead to a low-performance global model.Although this issue has been investigated by many previous works,this paper finds that they have little or no performance improvement over the standard baseline FedAvg when facing highly non-ⅡD data,unstable client participation,and deep models,seriously hindering the privacy protection application value of FL.To address this issue,a new solution called FedUp has been proposed.FedUp is a robust optimization solution for non-ⅡD FL that improves the generalization robustness of the global model while retaining the privacy protection characteristics of FL.FedUp minimizes the upper bound of the global empirical loss function to ensure that the models exhibit smaller generalization errors.Simulation experiments show that FedUp achieves significant advantages over state-of-the-art methods,and is robust to highly non-ⅡD data as well as unstable and large-cohort client participation.This solution has the potential to improve the performance of FL and make it more practical for privacy protection applications.
NETL
NSTL
万方数据
