Developing machine learning methods that are privacy preserving is today a central topic of research, with huge practical impacts. Among the numerous ways to address privacy-preserving learning, we here take the perspective of computing the divergences between distributions under the Differential Privacy (DP) framework -- being able to compute divergences between distributions is pivotal for many machine learning problems, such as learning generative models or domain adaptation problems. Instead of resorting to the popular gradient-based sanitization method for DP, we tackle the problem at its roots by focusing on the Sliced Wasserstein Distance and seamlessly making it differentially private. Our main contribution is as follows: we analyze the property of adding a Gaussian perturbation to the intrinsic randomized mechanism of the Sliced Wasserstein Distance, and we establish the sensitivityof the resulting differentially private mechanism. One of our important findings is that this DP mechanism transforms the Sliced Wasserstein distance into another distance, that we call the Smoothed Sliced Wasserstein Distance. This new differentially private distribution distance can be plugged into generative models and domain adaptation algorithms in a transparent way, and we empirically show that it yields highly competitive performance compared with gradient-based DP approaches from the literature, with almost no loss in accuracy for the domain adaptation problems that we consider.
翻译:隐私保护的机器教学方法的开发是当今研究的一个中心主题,具有巨大的实际影响。在解决隐私保护学习的众多方法中,我们在这里从以下角度出发,计算在差异隐私(DP)框架下的分布差异 -- -- 能够计算分配差异对于许多机器学习问题至关重要,例如学习基因模型或领域适应问题。我们没有为DP采用流行的基于梯度的消化方法,而是将这一问题从根源上解决,我们把注意力放在斯利切德·瓦瑟斯坦距离上,并且无缝地使其变得私人化。我们的主要贡献如下:我们分析在斯利切德·瓦瑟斯坦距离的内在随机化机制中增加一个高斯环绕的特性,我们确定由此产生的分配差异的敏感度,对于不同的私人机制,例如学习基因模型模型或领域适应问题。我们的一个重要发现,这种DP机制将斯利克特·瓦瑟斯坦距离转换为另一个距离,我们称之为滑度的斯利特·瓦瑟斯坦距离。这种新的私人分配差异性距离可以被插入到基因化模型和域内调。我们的主要贡献如下:我们分析如何在Slied Wasserstein refildalation commission compress,我们没有从一个透明化的方法中将它与具有高度的升级化,我们从一个比高等级化的指数化,我们从高分化的指数上将它与高度地研究。