自适应隐私预算分配的差分隐私梯度下降算法
Differential Privacy Gradient Descent Algorithm with Adaptive Privacy Budget Allocation
李界雯 1陈佳佳 1李师毅2
作者信息
- 1. 山西财经大学统计学院,山西 太原 030006
- 2. 山西大学计算机与信息技术学院,山西 太原 030006
- 折叠
摘要
在大数据时代的背景下利用机器学习模型解决问题的同时也存在着敏感数据泄露的安全风险.利用差分隐私方法对模型训练过程进行保护是解决该问题的有效方式,现有的差分隐私梯度下降算法虽然已经取得了显著的成果,但是在其迭代训练过程中依旧存在不足之处.文章从隐私预算分配的角度提出了 一种自适应选择学习率和隐私预算分配的差分隐私梯度下降算法.具体地,在保证模型收敛的前提下对每次迭代过程中的隐私预算进行自适应地分配,最大程度地对其进行隐私保护,防止模型参数的泄露.最后,实验结果验证了该算法的隐私性和有效性,并且在不同数据集中都表现出了优于已有算法的结果.
Abstract
In the era of big data,there are also security risks of sensitive data leakage when using machine learning models to solve problems.Using differential privacy method to protect the model training process is an effective way to solve this problem.Although the existing differential privacy gradient descent algorithm has achieved significant results,there are still shortcomings in its iterative training process.In this paper,we propose a differential privacy gradient descent algorithm for adaptively selecting the learning rate and privacy budget from the perspective of privacy budget allocation.Intuitively,at the beginning of the optimization,the gradients are expected to be large,so there is no need to measure them precisely.However,as the parameter approaches its optimal value,the gradient decreases and therefore needs to be measured more precisely.Finally,the experimental results verify the privacy and effectiveness of the proposed algorithm,and show better results than the existing algorithms in different data sets.
关键词
差分隐私/隐私预算分配/梯度下降/logistic回归Key words
differential privacy/privacy budget allocation/gradient descent/logistic regres-sion引用本文复制引用
基金项目
国家自然科学基金(62072293)
山西省基础研究计划项目(202103021223304)
山西省基础研究计划项目(202303021221054)
山西省研究生教育教学改革课题(2022YJJG010)
山西省高等学校教学改革创新项目(J20220570)
山西省回国留学人员科研教研资助项目(2024-002)
出版年
2024
NETL
NSTL
万方数据