Federated learning (FL) has attracted much attention as a privacy-preserving distributed machine learning framework, where many clients collaboratively train a machine learning model by exchanging model updates with a parameter server instead of sharing their raw data. Nevertheless, FL training suffers from slow convergence and unstable performance due to stragglers caused by the heterogeneous computational resources of clients and fluctuating communication rates. This paper proposes a coded FL framework to mitigate the straggler issue, namely stochastic coded federated learning (SCFL). In this framework, each client generates a privacy-preserving coded dataset by adding additive noise to the random linear combination of its local data. The server collects the coded datasets from all the clients to construct a composite dataset, which helps to compensate for the straggling effect. In the training process, the server as well as clients perform mini-batch stochastic gradient descent (SGD), and the server adds a make-up term in model aggregation to obtain unbiased gradient estimates. We characterize the privacy guarantee by the mutual information differential privacy (MI-DP) and analyze the convergence performance in federated learning. Besides, we demonstrate a privacy-performance tradeoff of the proposed SCFL method by analyzing the influence of the privacy constraint on the convergence rate. Finally, numerical experiments corroborate our analysis and show the benefits of SCFL in achieving fast convergence while preserving data privacy.
翻译:联邦学习(FL)作为一个保护隐私的分布式机器学习框架吸引了大量关注,在这个框架中,许多客户通过与参数服务器交换模型更新,而不是共享原始数据,对机器学习模式进行了合作培训;然而,FL培训由于客户的多种计算资源和波动通信率造成的累进效应造成累进效应,因而工作表现缓慢和不稳定;本文件提议了一个代码化FL框架,以缓解分流问题,即杂交编码的编码化化联邦学习(SCFL);在这个框架中,每个客户通过在本地数据的随机线性组合中添加添加添加添加噪音,从而生成了一个保存隐私的编码数据。服务器收集了所有客户的编码数据集,以构建综合数据集,这有助于弥补松散效应。在培训过程中,服务器和客户都进行了小型组合式组合式梯度梯度梯度梯度下降(SGD),服务器在模型汇总中增加了一个化术语,以获得公正的梯度估计。我们通过相互信息差异性隐私(MI-DP)以及分析反馈的整合性整合性业绩,同时通过反馈的保密性稳定性稳定化标准,展示我们提出的稳定性稳定化贸易趋一致率分析,从而最终展示了SLLFLL的升级分析。