We study the multi-agent Bayesian optimization (BO) problem, where multiple agents maximize a black-box function via iterative queries. We focus on Entropy Search (ES), a sample-efficient BO algorithm that selects queries to maximize the mutual information about the maximum of the black-box function. One of the main challenges of ES is that calculating the mutual information requires computationally-costly approximation techniques. For multi-agent BO problems, the computational cost of ES is exponential in the number of agents. To address this challenge, we propose the Gaussian Max-value Entropy Search, a multi-agent BO algorithm with favorable sample and computational efficiency. The key to our idea is to use a normal distribution to approximate the function maximum and calculate its mutual information accordingly. The resulting approximation allows queries to be cast as the solution of a closed-form optimization problem which, in turn, can be solved via a modified gradient ascent algorithm and scaled to a large number of agents. We demonstrate the effectiveness of Gaussian max-value Entropy Search through numerical experiments on standard test functions and real-robot experiments on the source-seeking problem. Results show that the proposed algorithm outperforms the multi-agent BO baselines in the numerical experiments and can stably seek the source with a limited number of noisy observations on real robots.
翻译:我们研究的是多试剂贝叶西亚优化(BO)问题, 即多个代理商通过迭接查询使黑盒功能最大化。 我们关注的是Entropy Search(ES),这是一个样本高效的BO算法,选择查询以最大限度地增加关于最大黑盒功能的相互信息。 ES的主要挑战之一是计算相互信息需要计算成本的近似技术。 对于多试剂BO问题,ES的计算成本在代理商数量上是指数指数化的。 为了应对这一挑战, 我们建议高西亚最大值 Entropy Search, 这是一种具有优异样本和计算效率的多试剂BO算法。 我们想法的关键是使用正常的分布来接近功能,并据此计算其相互信息。 由此产生的近似可以将查询作为封闭式优化问题的解决方案, 而后者反过来可以通过修正的梯度作为精度算法加以解决, 并推广到大量代理商的数量。 我们通过对标准测试功能进行数字实验, 和对来源- 机器人实验进行实际机器人实验, 并用限制的机器人实验, 数字- 测试, 显示提议的BOBA 结果可以找到一个数字级实验室的源- 。</s>