Bilevel optimization have gained growing interests, with numerous applications found in meta learning, minimax games, reinforcement learning, and nested composition optimization. This paper studies the problem of distributed bilevel optimization over a network where agents can only communicate with neighbors, including examples from multi-task, multi-agent learning and federated learning. In this paper, we propose a gossip-based distributed bilevel learning algorithm that allows networked agents to solve both the inner and outer optimization problems in a single timescale and share information via network propagation. We show that our algorithm enjoys the $\mathcal{O}(\frac{1}{K \epsilon^2})$ per-agent sample complexity for general nonconvex bilevel optimization and $\mathcal{O}(\frac{1}{K \epsilon})$ for strongly convex objective, achieving a speedup that scales linearly with the network size. The sample complexities are optimal in both $\epsilon$ and $K$. We test our algorithm on the examples of hyperparameter tuning and decentralized reinforcement learning. Simulated experiments confirmed that our algorithm achieves the state-of-the-art training efficiency and test accuracy.
翻译:双层优化在元学习、 迷你马克思游戏、 强化学习和嵌套成份优化中发现许多应用。 本文研究在代理商只能与邻居沟通的网络上分布双层优化的问题, 包括多任务、 多试剂学习和联合学习的例子。 在本文中, 我们提出基于八卦的双层优化算法, 使网络代理商能够在单一的时间尺度上解决内部和外部优化问题, 并通过网络传播共享信息。 我们显示我们的算法享有$\mathcal{ O}( afrac{ 1 ⁇ K\ \ epsilon% 2} ) 。 我们测试了用于一般非convex双层优化和 $\mathcal{O} (\frac{1 ⁇ K\ \ \ epsilon} ) 和 $\\\ mathcal} 的每个代理商样本复杂性。 测试证实了我们的效率测试和精度测试的精确性。