Many problems in science and engineering involve optimizing an expensive black-box function over a high-dimensional space. For such black-box optimization (BBO) problems, we typically assume a small budget for online function evaluations, but also often have access to a fixed, offline dataset for pretraining. Prior approaches seek to utilize the offline data to approximate the function or its inverse but are not sufficiently accurate far from the data distribution. We propose BONET, a generative framework for pretraining a novel black-box optimizer using offline datasets. In BONET, we train an autoregressive model on fixed-length trajectories derived from an offline dataset. We design a sampling strategy to synthesize trajectories from offline data using a simple heuristic of rolling out monotonic transitions from low-fidelity to high-fidelity samples. Empirically, we instantiate BONET using a causally masked Transformer and evaluate it on Design-Bench, where we rank the best on average, outperforming state-of-the-art baselines.
翻译:科学和工程方面的许多问题涉及在一个高维空间优化昂贵的黑匣子功能。 对于这种黑盒优化(BBO)问题,我们通常假定用于在线功能评估的预算很小,但往往可以使用固定的离线数据集进行预培训。 先前的方法是利用离线数据来接近功能或其反向,但距离数据分布不够准确。 我们提议BONET, 用于利用离线数据集对一个新的黑盒优化器进行预培训的基因化框架。 在BONET, 我们用离线数据集产生的固定长轨模型来培训自动递增模型。 我们设计了一个抽样战略, 利用从低不感官到高不感官样本的单向离线数据截图进行综合。 我们随机地使用因果化的掩码变换器对BONET进行速化, 并在设计- Bench 上对它进行评估, 在那里我们根据平均、超水平的状态基线进行最佳排序。