In a federated learning scenario where multiple parties jointly learn a model from their respective data, there exist two conflicting goals for the choice of appropriate algorithms. On one hand, private and sensitive training data must be kept secure as much as possible in the presence of \textit{semi-honest} partners, while on the other hand, a certain amount of information has to be exchanged among different parties for the sake of learning utility. Such a challenge calls for the privacy-preserving federated learning solution, which maximizes the utility of the learned model and maintains a provable privacy guarantee of participating parties' private data. This article illustrates a general framework that a) formulates the trade-off between privacy loss and utility loss from a unified information-theoretic point of view, and b) delineates quantitative bounds of privacy-utility trade-off when different protection mechanisms including Randomization, Sparsity, and Homomorphic Encryption are used. It was shown that in general \textit{there is no free lunch for the privacy-utility trade-off} and one has to trade the preserving of privacy with a certain degree of degraded utility. The quantitative analysis illustrated in this article may serve as the guidance for the design of practical federated learning algorithms.
翻译:在一个联合学习的情景中,多方共同从各自的数据中学习模型,在选择适当的算法方面有两个相互冲突的目标。一方面,私人和敏感培训数据必须尽可能在有\ textit{semi-honest}伙伴在场的情况下保持安全,另一方面,为了学习效用,不同方之间必须交流一定数量的信息。这种挑战要求采用保护隐私的联邦学习解决方案,最大限度地利用所学模型,并维持参与方私人数据的可变隐私保障。这一条展示了一个总体框架,即a)从统一的信息理论角度制定隐私损失和公用事业损失之间的权衡,b)在使用不同保护机制,包括随机化、分化和古典化加密时,界定隐私-效用交易的量化界限。在一般情况下,没有为隐私权-利用率交易提供免费午餐,而在实际效用交易中,有一个总框架,即从统一的信息理论角度制定隐私损失和效用损失之间的权衡,b)在使用不同保护机制时,包括随机化、分量化和单态加密时,界定隐私-效用交易的定量分析。量化显示,在一般情况下,没有为隐私-通用交易交易提供可自由午餐,而有一个总需进行交易的保密性交易,在实际效用设计中以某种节制化工具化工具化的节制设计中进行。