Asynchronous Federated Learning on Heterogeneous Devices: A Survey

Federated learning (FL) is experiencing a fast booming with the wave of distributed machine learning. In the FL paradigm, the global model is aggregated on the centralized aggregation server according to the parameters of local models instead of local training data, mitigating privacy leakage caused by the collection of sensitive information. With the increased computing and communication capabilities of edge and IoT devices, applying FL on heterogeneous devices to train machine learning models becomes a trend. The synchronous aggregation strategy in the classic FL paradigm cannot effectively use the limited resource, especially on heterogeneous devices, due to its waiting for straggler devices before aggregation in each training round. Furthermore, the disparity of data spread on devices (i.e. data heterogeneity) in real-world scenarios downgrades the accuracy of models. As a result, many asynchronous FL (AFL) paradigms are presented in various application scenarios to improve efficiency, performance, privacy, and security. This survey comprehensively analyzes and summarizes existing variants of AFL according to a novel classification mechanism, including device heterogeneity, data heterogeneity, privacy and security on heterogeneous devices, and applications on heterogeneous devices. Finally, this survey reveals rising challenges and presents potentially promising research directions in this under-investigated field.