Decentralized learning enables edge users to collaboratively train models by exchanging information via device-to-device communication, yet prior works have been limited to wireless networks with fixed topologies and reliable workers. In this work, we propose an asynchronous decentralized stochastic gradient descent (DSGD) algorithm, which is robust to the inherent computation and communication failures occurring at the wireless network edge. We theoretically analyze its performance and establish a non-asymptotic convergence guarantee. Experimental results corroborate our analysis, demonstrating the benefits of asynchronicity and outdated gradient information reuse in decentralized learning over unreliable wireless networks.
翻译:分散化学习使边缘用户能够通过设备对设备通信交流信息,合作培训模型,但先前的工程仅限于有固定地形和可靠工人的无线网络。在这项工作中,我们建议采用非同步分散式的分散式梯度梯度下降算法(DSGD),以适应无线网络边缘发生的内在计算和通信故障。我们从理论上分析其性能,并建立一个非零度趋同保证。实验结果证实了我们的分析,证明了无同步性和过时梯度信息在分散式学习而不可靠的无线网络方面的益处。