Federated Learning (FL) has shown great potential as a privacy-preserving solution to learning from decentralized data which are only accessible locally on end devices (i.e., clients). In many scenarios, however, a large proportion of the clients are probably in possession of low-quality data that are biased, noisy or even irrelevant. As a result, they could significantly slow down the convergence of the global model we aim to build and also compromise its quality. In light of this, we propose FedProf, a novel protocol for optimizing FL under such circumstances without breaching data privacy. The key of our approach is using the global model to dynamically profile the latent representations of data (termed representation footprints) on the clients. By matching local footprints on clients against a baseline footprint on the server, we adaptively score each client and adjust its probability of being selected each round so as to mitigate the impact of the clients with low-quality data on the training process. We have conducted extensive experiments on public data sets using various FL settings. The results show that FedProf effectively reduces the number of communication rounds and overall time (providing up to 4.5x speedup) for the global model to converge while improving the accuracy of the final global model.
翻译:联邦学习联合会(FL)已经展示出巨大的保护隐私的解决方案,从分散的数据(即客户)中学习,这种分散的数据只能在当地通过终端设备(即客户)获得。然而,在许多情形中,很大一部分客户可能拥有偏差、吵闹甚至无关紧要的低质量数据,因此,他们可以大大减缓我们所要建立的全球模型的趋同速度,并损害其质量。据此,我们提议FedProf,这是在这种情形下在不破坏数据隐私的情况下优化FL的一个新协议。我们的方法的关键是使用全球模型,动态地描述客户对数据的潜在显示(确定的代表足迹)。通过将客户的当地足迹与服务器的基线足迹相匹配,我们调整每个客户的得分,并调整其被选每轮的可能性,以减轻低质量数据客户对培训过程的影响。我们利用各种 FL环境对公共数据集进行了广泛的实验。结果显示,FedProf有效地减少了全球模型的最后模型的准确性,从而减少了通信回合和整个时间(达到4.5x速度)。