In this paper, we present an algorithm for accelerating decentralized stochastic gradient descent. Recently, decentralized stochastic optimization methods have attracted a lot of attention, mainly due to their low iteration cost, data locality and data exchange efficiency. They are generalizations of algorithms such as SGD and Local SGD. An additional important contribution of this work is the additions to the analysis of acceleration of stochastic methods, which allows achieving acceleration in the decentralized case.
翻译:在本文中,我们提出了一个加速分散的随机梯度下降的算法。 最近,分散的随机梯度优化方法吸引了大量关注,主要是因为其迭代成本、数据地点和数据交换效率较低。它们是SGD和本地 SGD等算法的概括。 这项工作的另一个重要贡献是对加速随机梯度方法分析的补充,这有利于在分散情况下实现加速。