Federated learning, in which training data is distributed among users and never shared, has emerged as a popular approach to privacy-preserving machine learning. Cryptographic techniques such as secure aggregation are used to aggregate contributions, like a model update, from all users. A robust technique for making such aggregates differentially private is to exploit infinite divisibility of the Laplace distribution, namely, that a Laplace distribution can be expressed as a sum of i.i.d. noise shares from a Gamma distribution, one share added by each user. However, Laplace noise is known to have suboptimal error in the low privacy regime for $\varepsilon$-differential privacy, where $\varepsilon > 1$ is a large constant. In this paper we present the first infinitely divisible noise distribution for real-valued data that achieves $\varepsilon$-differential privacy and has expected error that decreases exponentially with $\varepsilon$.
翻译:在用户之间分发培训数据,从未分享的联邦学习,已成为一种普及的保存隐私的机器学习方法。加密技术,例如安全汇总技术,被用来汇总来自所有用户的贡献,如一个模型更新。一种强大的私密技术是利用Laplace分布的无限分散性,即Laplace分布可以表现为id之和,即Gamma分布的噪音份额,每个用户增加一个份额。然而,Laplace噪音已知在低隐私制度中存在次优的错误,因为美元和瓦雷普西隆 > 1美元是一个很大的常数。我们在本论文中介绍了实现美元和瓦雷普西隆差异隐私的真正价值数据首次极不易分散的噪音分布,并预期错误会以美元和瓦雷普西隆美元以指数递减。