Federated learning (FL) allows multiple clients cooperatively train models without disclosing local data. However, the existing works fail to address all these practical concerns in FL: limited communication resources, dynamic network conditions and heterogeneous client properties, which slow down the convergence of FL. To tackle the above challenges, we propose a heterogeneity-aware FL framework, called FedCG, with adaptive client selection and gradient compression. Specifically, the parameter server (PS) selects a representative client subset considering statistical heterogeneity and sends the global model to them. After local training, these selected clients upload compressed model updates matching their capabilities to the PS for aggregation, which significantly alleviates the communication load and mitigates the straggler effect. We theoretically analyze the impact of both client selection and gradient compression on convergence performance. Guided by the derived convergence rate, we develop an iteration-based algorithm to jointly optimize client selection and compression ratio decision using submodular maximization and linear programming. Extensive experiments on both real-world prototypes and simulations show that FedCG can provide up to 5.3$\times$ speedup compared to other methods.
翻译:联邦学习(FL)允许多个客户在不披露当地数据的情况下合作培训模型。然而,现有的工程未能解决FL中所有这些实际问题:通信资源有限、动态网络条件和不同客户特性,这些都减缓了FL的趋同速度。为了应对上述挑战,我们提议了一个异质性-认知FL框架,称为FedCG,具有适应性客户选择和梯度压缩功能。具体地说,参数服务器选择了一个有代表性的客户子集,考虑到统计差异性,并将全球模型发送给它们。在当地培训之后,这些选定的客户上传压缩模型更新,使其能力与PS相匹配,从而大大减轻了通信负荷,减轻了递减效应。我们从理论上分析了客户选择和梯度压缩对趋同性效果的影响。根据衍生的趋同率,我们开发了基于迭代算法,以联合优化客户选择和压缩比率决定,使用亚调最大化和线性编程。关于现实世界原型和模拟的广泛实验显示FCCG可以提供高达5.3美元的时间,与其他方法相比。