Cross-silo federated learning utilizes a few hundred reliable data silos with high-speed access links to jointly train a model. While this approach becomes a popular setting in federated learning, designing a robust topology to reduce the training time is still an open problem. In this paper, we present a new multigraph topology for cross-silo federated learning. We first construct the multigraph using the overlay graph. We then parse this multigraph into different simple graphs with isolated nodes. The existence of isolated nodes allows us to perform model aggregation without waiting for other nodes, hence reducing the training time. We further propose a new distributed learning algorithm to use with our multigraph topology. The intensive experiments on public datasets show that our proposed method significantly reduces the training time compared with recent state-of-the-art topologies while ensuring convergence and maintaining the model's accuracy.
翻译:跨银河联盟学习利用数百个具有高速访问链接的可靠数据筒仓来联合训练模型。 虽然这个方法成为了联合学习的流行环境, 设计一个强大的地形学以减少培训时间仍然是一个尚未解决的问题。 在本文中, 我们提出一个新的跨银河联盟学习的多语种地形学。 我们首先使用重叠图构建多语种学。 我们然后用孤立节点将这个多语种分析成不同的简单图表。 孤立的节点的存在使我们能够在不等待其他节点的情况下进行模型集成, 从而减少培训时间。 我们进一步提出一个新的分布式学习算法, 与我们的多语种地形学一起使用。 在公共数据集上的密集实验表明, 我们提出的方法大大缩短了培训时间, 与最近的艺术状态表相比, 同时确保了模型的趋同性并保持准确性 。