Bayesian optimization (BO) is a popular paradigm for global optimization of expensive black-box functions, but there are many domains where the function is not completely a black-box. The data may have some known structure (e.g. symmetries) and/or the data generation process may be a composite process that yields useful intermediate or auxiliary information in addition to the value of the optimization objective. However, surrogate models traditionally employed in BO, such as Gaussian Processes (GPs), scale poorly with dataset size and do not easily accommodate known structure. Instead, we use Bayesian neural networks, a class of scalable and flexible surrogate models with inductive biases, to extend BO to complex, structured problems with high dimensionality. We demonstrate BO on a number of realistic problems in physics and chemistry, including topology optimization of photonic crystal materials using convolutional neural networks, and chemical property optimization of molecules using graph neural networks. On these complex tasks, we show that neural networks often outperform GPs as surrogate models for BO in terms of both sampling efficiency and computational cost.
翻译:Bayesian优化(BO)是全球优化昂贵黑盒功能的流行范例,但有许多领域,该功能并非完全是一个黑盒。数据可能有一些已知的结构(如对称)和/或数据生成过程可能是综合过程,除了优化目标的价值外,还产生有用的中间或辅助信息。然而,BO传统上采用的代用模型,如Gaussian进程(GPs),其规模与数据集大小不相称,不易容纳已知的结构。相反,我们使用Bayesian神经网络,这是一组具有直观偏差的可缩缩缩缩缩缩放和灵活替代模型,将BO扩大到复杂和结构化的问题,具有高维度。我们向BO展示了物理和化学方面的一些现实问题,包括利用电动神经网络对光晶材料进行表面优化,以及利用图形神经网络对分子进行化学属性优化。关于这些复杂任务,我们显示神经网络往往在取样效率和计算成本方面优于作为BO的替代模型。