Feature-based algorithm selection aims to automatically find the best one from a portfolio of optimization algorithms on an unseen problem based on its landscape features. Feature-based algorithm selection has recently received attention in the research field of black-box numerical optimization. However, algorithm selection for black-box optimization has been poorly understood. Most previous studies have focused only on whether an algorithm selection system can outperform the single-best solver in a portfolio. In addition, a benchmarking methodology for algorithm selection systems has not been well investigated in the literature. In this context, this paper analyzes algorithm selection systems on the 24 noiseless black-box optimization benchmarking functions. First, we demonstrate that the successful performance 1 measure is more reliable than the expected runtime measure for benchmarking algorithm selection systems. Then, we examine the influence of randomness on the performance of algorithm selection systems. We also show that the performance of algorithm selection systems can be significantly improved by using a pre-solver. We point out that the difficulty of outperforming the single-best solver depends on algorithm portfolios, cross-validation methods, and dimensions. Finally, we demonstrate that the effectiveness of algorithm portfolios depends on various factors. These findings provide fundamental insights for algorithm selection for black-box optimization.
翻译:以功能为基础的算法选择旨在基于其景观特征,从一个隐蔽问题的优化算法组合中自动找到最佳的。 以特性为基础的算法选择最近在黑盒数字优化的研究领域受到关注。 但是,黑盒优化的算法选择没有很好理解。 先前的大多数研究仅侧重于一个算法选择系统能否在组合中优于一个最佳求解器。 此外, 文献没有很好地调查算法选择系统的基准方法。 在这方面, 本文分析了24个无噪音黑盒优化基准功能的算法选择系统。 首先, 我们证明, 成功的业绩1衡量比基准算法选择系统的预期运行时间尺度更加可靠。 然后, 我们检查随机性对算法选择系统性的影响。 我们还表明, 使用一个预解算法选择系统可以大大改进算法选择系统的性能。 我们指出, 单项选择法选择方法的难度取决于算法组合、 交叉验证方法和维度。 最后, 我们证明, 算法组合的有效性取决于各种因素。 这些结论提供了最根本的算法选择。