Bayesian optimization is a popular method for optimizing expensive black-box functions. The objective functions of hard real world problems are oftentimes characterized by a fluctuated landscape of many local optima. Bayesian optimization risks in over-exploiting such traps, remaining with insufficient query budget for exploring the global landscape. We introduce Coordinate Backoff Bayesian optimization (CobBO) to alleviate those challenges. CobBO captures a smooth approximation of the global landscape by interpolating the values of queried points projected to randomly selected promising coordinate subspaces. Thus also a smaller query budget is required for the Gaussian process regressions applied over the lower dimensional subspaces. This approach can be viewed as a variant of coordinate ascent, tailored for Bayesian optimization, using a stopping rule for backing off from a certain subspace and switching to another coordinate subset. Additionally, adaptive trust regions are dynamically formed to expedite the convergence, and stagnant local optima are escaped by switching trust regions. Further smoothness and acceleration are achieved by filtering out clustered queried points. Through comprehensive evaluations over a wide spectrum of benchmarks, CobBO is shown to consistently find comparable or better solutions, with a reduced trial complexity compared to the state-of-the-art methods in both low and high dimensions.
翻译:Bayesian 优化是优化昂贵黑盒功能的流行方法。 硬现实世界问题的目标功能往往以许多本地奥普度地貌波动为特征。 Bayesian 在过度开发这些陷阱时的优化风险, 仍然没有足够的探索全球地貌的查询预算。 我们引入了协调的后退巴伊西亚优化( CobbBO) 来缓解这些挑战。 CobBO 通过将随机选择的有希望协调的子空间所预测的点的数值乘以随机选择的有希望协调的子空间, 来捕捉全球地貌的平稳近似。 因此, 在低维度子空间上应用的高频进程回归也需要一个较小的查询预算。 这种方法可以被视为一种协调的变体, 适合Bayesian 优化, 使用一条停止从某个子空间退缩的规则, 并转换到另一个协调子块来缓解这些挑战。 此外, 适应性信任区域是动态形成的, 通过转换信任区域来避免停滞的本地optima。 通过过滤组合式查询点实现进一步的平稳和加速。 通过对广泛的基准范围进行综合评估, CobBO 显示, 以持续的复杂度与较低度相比的方法。