Federated learning is a method of training a global model from decentralized data distributed across client devices. Here, model parameters are computed locally by each client device and exchanged with a central server, which aggregates the local models for a global view, without requiring sharing of training data. The convergence performance of federated learning is severely impacted in heterogeneous computing platforms such as those at the wireless edge, where straggling computations and communication links can significantly limit timely model parameter updates. This paper develops a novel coded computing technique for federated learning to mitigate the impact of stragglers. In the proposed Coded Federated Learning (CFL) scheme, each client device privately generates parity training data and shares it with the central server only once at the start of the training phase. The central server can then preemptively perform redundant gradient computations on the composite parity data to compensate for the erased or delayed parameter updates. Our results show that CFL allows the global model to converge nearly four times faster when compared to an uncoded approach
翻译:联邦学习是一种方法,用于从客户设备之间分布的分散数据中培训全球模型。 这里, 模型参数由每个客户设备在当地计算, 并与中央服务器交换, 中央服务器将本地模型汇总为全球观点, 不需要共享培训数据 。 联合学习的趋同性性能在无线边缘等多式计算平台中受到严重影响, 在无线边缘, 折叠计算和通信链接可以大大限制及时的模型参数更新 。 本文开发了一种新的混合学习编码计算技术, 以缓解拖动器的影响 。 在拟议的编码联邦学习( CFL) 方案中, 每个客户设备在培训阶段开始时只私下生成对等培训数据, 并且只与中央服务器共享一次 。 中央服务器然后可以先对综合对等数据进行重复的梯度计算, 以补偿被删除或延迟的参数更新 。 我们的结果表明, CFL允许全球模型在与未编码方法相比, 近四倍的趋同速度。