Federated learning (FL) has emerged as an instance of distributed machine learning paradigm that avoids the transmission of data generated on the users' side. Although data are not transmitted, edge devices have to deal with limited communication bandwidths, data heterogeneity, and straggler effects due to the limited computational resources of users' devices. A prominent approach to overcome such difficulties is FedADMM, which is based on the classical two-operator consensus alternating direction method of multipliers (ADMM). The common assumption of FL algorithms, including FedADMM, is that they learn a global model using data only on the users' side and not on the edge server. However, in edge learning, the server is expected to be near the base station and have direct access to rich datasets. In this paper, we argue that leveraging the rich data on the edge server is much more beneficial than utilizing only user datasets. Specifically, we show that the mere application of FL with an additional virtual user node representing the data on the edge server is inefficient. We propose FedTOP-ADMM, which generalizes FedADMM and is based on a three-operator ADMM-type technique that exploits a smooth cost function on the edge server to learn a global model parallel to the edge devices. Our numerical experiments indicate that FedTOP-ADMM has substantial gain up to 33\% in communication efficiency to reach a desired test accuracy with respect to FedADMM, including a virtual user on the edge server.
翻译:联邦学习(FL)已经成为分散式机器学习模式的范例,避免传输用户方面产生的数据。虽然数据没有传输,但边缘设备必须处理有限的通信带宽、数据异质性和因用户设备计算资源有限而产生的分流效应。一个突出的克服这些困难的方法是FedADMMM, 其基础是传统的双操作者共识乘数交错方向法(ADMM); FL算法的共同假设,包括FDADMMM, 其共同假设是,他们学习了一个仅使用用户方面而非边缘服务器数据的全球模型。然而,在边缘学习中,服务器预计将接近基站,并直接接触丰富的数据集。在本文中,我们认为,利用边缘服务器上的丰富数据比仅仅使用用户数据集(ADMMMM)更有益。 具体地说,仅仅应用FL(包括FDADMMMMMM ) 代表边缘服务器数据的额外虚拟用户节点是低效的。我们建议FDTOP-ADMMMMM, 将FAMMMMM-S的精度测试功能与MD平坦性升级到3MAD-ADADADAD 工具。