Federated Learning (FL) is a novel machine learning framework, which enables multiple distributed devices cooperatively training a shared model scheduled by a central server while protecting private data locally. However, the non-independent-and-identically-distributed (Non-IID) data samples and frequent communication among participants will slow down the convergent rate and increase communication costs. To achieve fast convergence, we ameliorate the local gradient descend approach in conventional local update rule by introducing the aggregated gradients at each local update epoch, and propose an adaptive learning rate algorithm that further takes the deviation of local parameter and global parameter into consideration at each iteration. The above strategy requires all clients' local parameters and gradients at each local iteration, which is challenging as there is no communication during local update epochs. Accordingly, we utilize mean field approach by introducing two mean field terms to estimate the average local parameters and gradients respectively, which does not require clients to exchange their private information with each other at each local update epoch. Numerical results show that our proposed framework is superior to the state-of-art schemes in model accuracy and convergent rate on both IID and Non-IID dataset.
翻译:联邦学习(FL)是一种新颖的机器学习框架,它使多个分布式设备可以合作地训练一个由中央服务器调度的共享模型,同时保护本地私有数据。然而,非独立同分布(Non-IID)数据样本和参与者之间的频繁通信会减缓收敛速度并增加通信开销。为了实现快速收敛,我们改善了传统本地更新规则中的局部梯度下降方法,通过引入聚合梯度来提出策略,并在每次迭代中进一步考虑局部参数和全局参数的偏差而提出自适应学习率算法。上述策略需要在每次本地更新时获得所有客户端的本地参数和梯度情况,但是在本地更新轮次中没有通信,这是具有挑战性的。因此,我们利用均值场方法,通过引入两个均值场项来估计平均本地参数和梯度,这不需要客户端在每次本地更新轮次中彼此交换他们的私有信息。数值结果显示,我们提出的框架在IID和非IID数据集上的模型准确度和收敛速率方面优于当前最先进的方案。