The optimal design of federated learning (FL) algorithms for solving general machine learning (ML) problems in practical edge computing systems with quantized message passing remains an open problem. This paper considers an edge computing system where the server and workers have possibly different computing and communication capabilities and employ quantization before transmitting messages. To explore the full potential of FL in such an edge computing system, we first present a general FL algorithm, namely GenQSGD, parameterized by the numbers of global and local iterations, mini-batch size, and step size sequence. Then, we analyze its convergence for an arbitrary step size sequence and specify the convergence results under three commonly adopted step size rules, namely the constant, exponential, and diminishing step size rules. Next, we optimize the algorithm parameters to minimize the energy cost under the time constraint and convergence error constraint, with the focus on the overall implementing process of FL. Specifically, for any given step size sequence under each considered step size rule, we optimize the numbers of global and local iterations and mini-batch size to optimally implement FL for applications with preset step size sequences. We also optimize the step size sequence along with these algorithm parameters to explore the full potential of FL. The resulting optimization problems are challenging non-convex problems with non-differentiable constraint functions. We propose iterative algorithms to obtain KKT points using general inner approximation (GIA) and tricks for solving complementary geometric programming (CGP). Finally, we numerically demonstrate the remarkable gains of GenQSGD with optimized algorithm parameters over existing FL algorithms and reveal the significance of optimally designing general FL algorithms.
翻译:为解决实际边缘计算机系统中的通用机器学习(ML)问题而最优化地设计FL学习(FL)算法,即GenQSGD,该算法以全球和地方迭代数、小批量尺寸和步数序列为参数。然后,我们分析其趋同任意的步数序列,并具体说明在三种常用的步数大小规则下,即恒定、指数和缩小步数规则下的趋同结果。本文认为,在发送信息之前,服务器和工人可能具有不同的计算和通信能力,并采用四舍五入法。为了在这种边缘计算系统中探索FL的全部潜力,我们首先将FL全部的算法值参数优化,然后将FL总级的级数序列和小批数尺寸的大小与最优化地执行FLL的应用程序相匹配。我们用预定的步数序列序列来优化FL的FL值,然后将最终的级数序列与FL级数比值进行最优化的缩数,我们用FL的平级的平级定的平级标准来优化地算。