Online federated learning (FL) enables geographically distributed devices to learn a global shared model from locally available streaming data. Most online FL literature considers a best-case scenario regarding the participating clients and the communication channels. However, these assumptions are often not met in real-world applications. Asynchronous settings can reflect a more realistic environment, such as heterogeneous client participation due to available computational power and battery constraints, as well as delays caused by communication channels or straggler devices. Further, in most applications, energy efficiency must be taken into consideration. Using the principles of partial-sharing-based communications, we propose a communication-efficient asynchronous online federated learning (PAO-Fed) strategy. By reducing the communication overhead of the participants, the proposed method renders participation in the learning task more accessible and efficient. In addition, the proposed aggregation mechanism accounts for random participation, handles delayed updates and mitigates their effect on accuracy. We prove the first and second-order convergence of the proposed PAO-Fed method and obtain an expression for its steady-state mean square deviation. Finally, we conduct comprehensive simulations to study the performance of the proposed method on both synthetic and real-life datasets. The simulations reveal that in asynchronous settings, the proposed PAO-Fed is able to achieve the same convergence properties as that of the online federated stochastic gradient while reducing the communication overhead by 98 percent.
翻译:在线联邦学习(FL)使地理分散的设备能够从本地可用的数据流中学习全局共享模型。大多数在线FL文献都考虑了参与客户和通信渠道的最佳情况。但是,这些假设在实际应用中往往无法满足。异步设置可以反映更现实的环境,例如由于可用的计算功率和电池约束而导致的异构客户端参与,以及通信渠道或迟到的设备导致的延迟。此外,在大多数应用中,必须考虑能源效率。使用部分共享通信的原则,我们提出了一种通信高效的异步在线联邦学习(PAO-Fed)策略。通过减少参与者的通信开销,提出的方法使参与学习任务更加可访问和高效。此外,所提出的聚合机制考虑随机参与,处理延迟更新并减轻其对准确性的影响。我们证明了所提出的PAO-Fed方法的一阶和二阶收敛,并获得其稳态均方差的表达式。最后,我们进行了全面的模拟,以研究所提出的方法在合成和现实生活数据集上的性能。模拟表明,在异步设置中,所提出的PAO-Fed能够实现与在线联邦随机梯度相同的收敛性能,同时将通信开销降低了98%。