We consider federated learning in tiered communication networks. Our network model consists of a set of silos, each holding a vertical partition of the data. Each silo contains a hub and a set of clients, with the silo's vertical data shard partitioned horizontally across its clients. We propose Tiered Decentralized Coordinate Descent (TDCD), a communication-efficient decentralized training algorithm for such two-tiered networks. The clients in each silo perform multiple local gradient steps before sharing updates with their hub to reduce communication overhead. Each hub adjusts its coordinates by averaging its workers' updates, and then hubs exchange intermediate updates with one another. We present a theoretical analysis of our algorithm and show the dependence of the convergence rate on the number of vertical partitions and the number of local updates. We further validate our approach empirically via simulation-based experiments using a variety of datasets and objectives.
翻译:我们考虑在分层通信网络中进行联合学习。 我们的网络模型由一组筒仓组成, 每个筒仓都持有数据垂直分割。 每个筒仓都包含一个枢纽和一组客户, 筒仓的垂直数据是垂直分割的, 其客户是横向分割的。 我们提议为这种分层网络使用一个通信效率高的分层协调源( TDCD), 这是一种通信效率高的分层培训算法。 每个筒仓的客户在与中心共享更新信息以降低通信间接费用之前, 执行多个本地梯度步骤。 每个枢纽通过平均其工人更新信息来调整其坐标, 然后中心中心相互交换中间更新信息。 我们对我们的算法进行理论分析, 并显示集率对垂直分割数和地方更新数的依赖性。 我们通过使用各种数据集和目标的模拟实验进一步验证我们的方法。