FedProf: Efficient Federated Learning with Data Representation Profiling

Federated Learning (FL) has shown great potential as a privacy-preserving solution to learning from decentralized data which are only accessible locally on end devices (i.e., clients). In many scenarios, however, a large proportion of the clients are probably in possession of low-quality data that are biased, noisy or even irrelevant. As a result, they could significantly slow down the convergence of the global model we aim to build and also compromise its quality. In light of this, we propose FedProf, a novel protocol for optimizing FL under such circumstances without breaching data privacy. The key of our approach is using the global model to dynamically profile the latent representations of data (termed representation footprints) on the clients. By matching local footprints on clients against a baseline footprint on the server, we adaptively score each client and adjust its probability of being selected each round so as to mitigate the impact of the clients with low-quality data on the training process. We have conducted extensive experiments on public data sets using various FL settings. The results show that FedProf effectively reduces the number of communication rounds and overall time (providing up to 4.5x speedup) for the global model to converge while improving the accuracy of the final global model.