In the paper, we propose an effective and efficient Compositional Federated Learning (ComFedL) algorithm for solving a new compositional Federated Learning (FL) framework, which frequently appears in many data mining and machine learning problems with a hierarchical structure such as distributionally robust FL and model-agnostic meta learning (MAML). Moreover, we study the convergence analysis of our ComFedL algorithm under some mild conditions, and prove that it achieves a convergence rate of $O(\frac{1}{\sqrt{T}})$, where $T$ denotes the number of iteration. To the best of our knowledge, our new Compositional FL framework is the first work to bridge federated learning with composition stochastic optimization. In particular, we first transform the distributionally robust FL (i.e., a minimax optimization problem) into a simple composition optimization problem by using KL divergence regularization. At the same time, we also first transform the distribution-agnostic MAML problem (i.e., a minimax optimization problem) into a simple yet effective composition optimization problem. Finally, we apply two popular machine learning tasks, i.e., distributionally robust FL and MAML to demonstrate the effectiveness of our algorithm.
翻译:在本文中,我们提出了解决新的组成联邦学习(FL)框架的有效和高效的联邦学习(ComFedL)算法(ComFedL ), 解决新的组成联邦学习(FL)框架。 在许多数据挖掘和机器学习问题中,许多数据采集和机器学习问题经常出现,如分布式强的FL和模型-不可知性元学习(MAML )等等级结构。 此外,我们还在一些温和的条件下研究我们的ComFedL算法(ComFedL)的趋同分析,并证明它达到了美元(frac{1unsqrt{T ⁇ )的趋同率($T$T$)的趋同率,用美元表示迭代数的数量。 根据我们的知识,我们新的组成FL框架是首次将联合学习与构成优化联系起来。特别是,我们首先通过使用 KL 差异调节,将分配- AMPL (e) 和 MAL (i. MAL) 两种大众学习任务转变为简单有效的FL 。