Most works on federated learning (FL) focus on the most common frequentist formulation of learning whereby the goal is minimizing the global empirical loss. Frequentist learning, however, is known to be problematic in the regime of limited data as it fails to quantify epistemic uncertainty in prediction. Bayesian learning provides a principled solution to this problem by shifting the optimization domain to the space of distribution in the model parameters. {\color{black}This paper proposes a novel mechanism for the efficient implementation of Bayesian learning in wireless systems. Specifically, we focus on a standard gradient-based Markov Chain Monte Carlo (MCMC) method, namely Langevin Monte Carlo (LMC), and we introduce a novel protocol, termed Wireless Federated LMC (WFLMC), that is able to repurpose channel noise for the double role of seed randomness for MCMC sampling and of privacy preservation.} To this end, based on the analysis of the Wasserstein distance between sample distribution and global posterior distribution under privacy and power constraints, we introduce a power allocation strategy as the solution of a convex program. The analysis identifies distinct operating regimes in which the performance of the system is power-limited, privacy-limited, or limited by the requirement of MCMC sampling. Both analytical and simulation results demonstrate that, if the channel noise is properly accounted for under suitable conditions, it can be fully repurposed for both MCMC sampling and privacy preservation, obtaining the same performance as in an ideal communication setting that is not subject to privacy constraints.
翻译:大部分关于联邦学习(FL)的工作都侧重于最常见的常见学习模式,即:该目标是最大限度地减少全球经验损失。然而,常见学习在有限的数据制度中是有问题的,因为它未能量化预测中隐含的不确定性。巴伊西亚学习通过将优化域转移到模型参数的分布空间,为这一问题提供了一个原则性的解决办法。 ~color{black}本文提出了在无线系统中高效实施巴伊西亚学习的新机制。具体地说,我们侧重于基于标准梯度的Markov 链 蒙特卡洛(MCMC)方法,即Langevin Monte Carlo(LMC),我们引入了称为Wireless United LMC(WFLMC)的新协议,因为它无法在预测中量化噪音。 为了将种子随机采样和隐私保护的双重作用,可以重新定位噪音。 ~为此,根据对抽样和权力分布在隐私和权力限制下,我们引入权力分配战略,作为连接程序的解决办法。分析发现,在精确度方面,在精确度和精确度方面,在精确度方面,在精确度方面,在精确度方面,在精确度方面,在精确度方面,在精确度方面,在精确度方面,在精确度方面,在精确度方面,在精确度方面,对精确度方面,对精确度和精确度方面,对精确性分析结果进行。