Differentially private training offers a protection which is usually interpreted as a guarantee against membership inference attacks. By proxy, this guarantee extends to other threats like reconstruction attacks attempting to extract complete training examples. Recent works provide evidence that if one does not need to protect against membership attacks but instead only wants to protect against training data reconstruction, then utility of private models can be improved because less noise is required to protect against these more ambitious attacks. We investigate this further in the context of DP-SGD, a standard algorithm for private deep learning, and provide an upper bound on the success of any reconstruction attack against DP-SGD together with an attack that empirically matches the predictions of our bound. Together, these two results open the door to fine-grained investigations on how to set the privacy parameters of DP-SGD in practice to protect against reconstruction attacks. Finally, we use our methods to demonstrate that different settings of the DP-SGD parameters leading to the same DP guarantees can result in significantly different success rates for reconstruction, indicating that the DP guarantee alone might not be a good proxy for controlling the protection against reconstruction attacks.
翻译:不同的私人培训提供了一种保护,通常被解释为对会员资格攻击的保障。通过代理,这种保障扩展到其他威胁,如试图提取完整培训实例的重建攻击等。最近的工作提供了证据,证明如果一个人不需要保护成员免受攻击,而只是希望保护不受培训数据重建的伤害,那么私人模式的效用就可以得到改善,因为需要减少噪音,以保护免受这些更加雄心勃勃的攻击。我们在DP-SGD这一私人深层学习的标准算法的背景下进一步调查了这一点,并为对DP-SGD的任何重建攻击的成功提供了上限,同时提供了在经验上符合我们约束预测的攻击。 这两项结果共同打开了对如何制定DP-SGD隐私参数以实际防范重建攻击的精细调查的大门。 最后,我们用我们的方法表明,导致同样的DP-SGD参数的不同环境可能导致显著不同的重建成功率,表明只有DP保证本身可能不是控制重建攻击的好替代物。