We consider Bayesian optimization of the output of a network of functions, where each function takes as input the output of its parent nodes, and where the network takes significant time to evaluate. Such problems arise, for example, in reinforcement learning, engineering design, and manufacturing. While the standard Bayesian optimization approach observes only the final output, our approach delivers greater query efficiency by leveraging information that the former ignores: intermediate output within the network. This is achieved by modeling the nodes of the network using Gaussian processes and choosing the points to evaluate using, as our acquisition function, the expected improvement computed with respect to the implied posterior on the objective. Although the non-Gaussian nature of this posterior prevents computing our acquisition function in closed form, we show that it can be efficiently maximized via sample average approximation. In addition, we prove that our method is asymptotically consistent, meaning that it finds a globally optimal solution as the number of evaluations grows to infinity, thus generalizing previously known convergence results for the expected improvement. Notably, this holds even though our method might not evaluate the domain densely, instead leveraging problem structure to leave regions unexplored. Finally, we show that our approach dramatically outperforms standard Bayesian optimization methods in several synthetic and real-world problems.
翻译:我们认为巴耶斯优化功能网络的产出,每个功能以其母节点的输出为输入,而网络需要大量时间进行评估。这些问题出现于强化学习、工程设计和制造等方面。标准的巴耶斯优化方法只观察最终产出,而我们的方法则通过利用前者忽略的信息来提高查询效率:网络内部的中间产出。这是通过利用高萨进程对网络节点进行建模,并选择点来评估,将先前已知的趋同结果作为我们获取功能,从而将目标上隐含的后遗迹的预期改进用于评估。尽管这个后遗迹的非加西文性质无法以封闭的形式计算我们的购置功能,但我们表明,通过样本平均近似,它可以高效地实现最大程度的获取。此外,我们证明我们的方法是过于同步的,意味着它发现全球最佳的解决方案,因为评价的数量越来越不尽相同,从而将先前已知的趋同结果推广到预期的改进。值得注意的是,尽管我们的方法可能不评估区域域的高度密度,而是将问题推向最终的海边际结构显示我们真正的合成方法。