Bayesian optimization (BO) has become popular for sequential optimization of black-box functions. When BO is used to optimize a target function, we often have access to previous evaluations of potentially related functions. This begs the question as to whether we can leverage these previous experiences to accelerate the current BO task through meta-learning (meta-BO), while ensuring robustness against potentially harmful dissimilar tasks that could sabotage the convergence of BO. This paper introduces two scalable and provably robust meta-BO algorithms: robust meta-Gaussian process-upper confidence bound (RM-GP-UCB) and RM-GP-Thompson sampling (RM-GP-TS). We prove that both algorithms are asymptotically no-regret even when some or all previous tasks are dissimilar to the current task, and show that RM-GP-UCB enjoys a better theoretical robustness than RM-GP-TS. We also exploit the theoretical guarantees to optimize the weights assigned to individual previous tasks through regret minimization via online learning, which diminishes the impact of dissimilar tasks and hence further enhances the robustness. Empirical evaluations show that (a) RM-GP-UCB performs effectively and consistently across various applications, and (b) RM-GP-TS, despite being less robust than RM-GP-UCB both in theory and in practice, performs competitively in some scenarios with less dissimilar tasks and is more computationally efficient.
翻译:Bayesian 优化 Bayesian Bayesian (BO) 已经为黑箱功能的顺序优化而流行。 当BO被用于优化目标功能时,我们常常有机会获得以前对潜在相关功能的评估。 这就提出了这样一个问题,即我们是否能够利用这些以往的经验,通过元学习(meta-BO)来加快当前 BBO的任务,同时确保稳健应对可能破坏BO趋同的潜在有害的不同任务。本文提出了两种可伸缩和可察觉到的稳健的元BOA算法:强的元-Gaussian进程增强信心(RM-GP-UB)和RMG-G-GP-TS抽样(RG-G-GP-G-TS),通过网上学习减少不同任务的影响,从而进一步加强了不相近的任务的影响,从而进一步强化了RMM-G-G-G-G-G-G-G-G-G-L-A的稳性做法,而不是持续地进行不稳健的(B-GM-G-G-G-G-C-C-C-C-C-C-C-CUD-C-C-C-C-C-D-C-C-C-CUDRMD-D-D-C-C-DRBD-C-D-D-D-D-C-C-D-C-C-D-C-C-D-C-C-C-D-D-D-D-D-C-C-C-C-C-C-C-C-C-DRMDRMD-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-D-D-D-D-D-D-D-D-D-D-C-DADBDADADADADADADADADADADADADAD-D-DADADADAD-DADADADADADADADADADADADAD-D-DAD-DAD-D-D