High-throughput Simulation of Federated Learning via Resource-Aware Client Placement

Federated Learning (FL) is the privacy-preserving machine learning paradigm which collaboratively trains a model across millions of devices. Simulated environments are fundamental to large-scale FL research, allowing researchers to quickly test new ideas to solve system and statistical heterogeneity issues. This work proposes \emph{Pollen}, a novel resource-aware system capable of speeding up FL simulations by efficiently placing clients across distributed and heterogeneous hardware. We propose minimising server-GPU communication and using an efficient client placement policy based on the inherent trade-offs of FL client placement on heterogeneous GPUs. These trade-offs are explored experimentally. This exploration has been conducted via relevant baselines on three popular FL tasks: image classification, speech recognition and text generation. We compare \emph{Pollen} to existing ad-hoc FL frameworks, such as Flower, Flute and FedScale, and show performance gains of $50\%$ to $400\%$.