Collaborative training of neural networks leverages distributed data by exchanging gradient information between different clients. Although training data entirely resides with the clients, recent work shows that training data can be reconstructed from such exchanged gradient information. To enhance privacy, gradient perturbation techniques have been proposed. However, they come at the cost of reduced model performance, increased convergence time, or increased data demand. In this paper, we introduce PRECODE, a PRivacy EnhanCing mODulE that can be used as generic extension for arbitrary model architectures. We propose a simple yet effective realization of PRECODE using variational modeling. The stochastic sampling induced by variational modeling effectively prevents privacy leakage from gradients and in turn preserves privacy of data owners. We evaluate PRECODE using state of the art gradient inversion attacks on two different model architectures trained on three datasets. In contrast to commonly used defense mechanisms, we find that our proposed modification consistently reduces the attack success rate to 0% while having almost no negative impact on model training and final performance. As a result, PRECODE reveals a promising path towards privacy enhancing model extensions.


翻译:虽然培训数据完全属于客户,但最近的工作表明,培训数据可以从这种交换的梯度信息中重建。为了加强隐私,提出了梯度扰动技术,但是,这些技术是以模型性能降低、趋同时间增加或数据需求增加为代价的。在本文件中,我们引入了可用作任意模型结构通用扩展的PRECODE,即PRECODE,即可用作攻击成功率普利维西的模范机制。我们建议使用变式模型来简单而有效地实现PRECODE。由于变式模型的诱发的随机抽样有效地防止了梯度的隐私渗漏,从而保护了数据拥有者的隐私。我们用在三个数据集上培训的两种不同的模型的艺术梯度攻击对PRECODE进行了评估。与常用的防御机制相比,我们发现我们提议的修改始终将攻击成功率降低到0%,同时对模型培训和最后性能产生几乎没有消极影响。结果,PRECODE显示了加强隐私模式扩展的一条有希望的道路。

0
下载
关闭预览

相关内容

iOS 8 提供的应用间和应用跟系统的功能交互特性。
  • Today (iOS and OS X): widgets for the Today view of Notification Center
  • Share (iOS and OS X): post content to web services or share content with others
  • Actions (iOS and OS X): app extensions to view or manipulate inside another app
  • Photo Editing (iOS): edit a photo or video in Apple's Photos app with extensions from a third-party apps
  • Finder Sync (OS X): remote file storage in the Finder with support for Finder content annotation
  • Storage Provider (iOS): an interface between files inside an app and other apps on a user's device
  • Custom Keyboard (iOS): system-wide alternative keyboards

Source: iOS 8 Extensions: Apple’s Plan for a Powerful App Ecosystem
专知会员服务
50+阅读 · 2020年12月14日
专知会员服务
44+阅读 · 2020年10月31日
最新【深度生成模型】Deep Generative Models,104页ppt
专知会员服务
68+阅读 · 2020年10月24日
【Google】平滑对抗训练,Smooth Adversarial Training
专知会员服务
48+阅读 · 2020年7月4日
已删除
将门创投
3+阅读 · 2019年4月19日
Arxiv
6+阅读 · 2020年10月8日
Arxiv
4+阅读 · 2018年4月26日
VIP会员
相关资讯
已删除
将门创投
3+阅读 · 2019年4月19日
Top
微信扫码咨询专知VIP会员