Multi-server Federated learning (FL) has been considered as a promising solution to address the limited communication resource problem of single-server FL. We consider a typical multi-server FL architecture, where the coverage areas of regional servers may overlap. The key point of this architecture is that the clients located in the overlapping areas update their local models based on the average model of all accessible regional models, which enables indirect model sharing among different regional servers. Due to the complicated network topology, the convergence analysis is much more challenging than single-server FL. In this paper, we firstly propose a novel MS-FedAvg algorithm for this multi-server FL architecture and analyze its convergence on non-iid datasets for general non-convex settings. Since the number of clients located in each regional server is much less than in single-server FL, the bandwidth of each client should be large enough to successfully communicate training models with the server, which indicates that full client participation can work in multi-server FL. Also, we provide the convergence analysis of the partial client participation scheme and develop a new biased partial participation strategy to further accelerate convergence. Our results indicate that the convergence results highly depend on the ratio of the number of clients in each area type to the total number of clients in all three strategies. The extensive experiments show remarkable performance and support our theoretical results.
翻译:多观测器联合会学习(FL)被认为是解决单一观测器FL有限通信资源问题的有希望的解决办法。我们认为,这是一个典型的多观测器FL结构,区域服务器的覆盖范围可能重叠。这一结构的关键点是,位于重叠地区的客户根据所有无障碍区域模型的平均模式更新其本地模型,使不同区域服务器之间能够间接共享模型。由于网络地形复杂,趋同分析比单一服务器FL更具挑战性。在本文中,我们首先为这一多观测器FL结构提出一个新的MS-FedAvg算法,并分析其在一般非电离层设置的非二数据集上的趋同性。由于每个区域服务器的客户数目远远少于单一服务器FL,每个客户的带宽应该足够大,以便成功地与服务器交流培训模型,这表明客户的充分参与可以在多观测器FL中发挥作用。此外,我们为这一多观测器结构提供了对部分客户参与计划的趋同性分析,并为每个客户的高度趋同性战略制定了新的偏差部分参与战略,从而进一步加速了我们三大客户的趋同性实验的结果。