In order to allow for large-scale, landscape-aware, per-instance algorithm selection, a benchmarking platform software is key. IOHexperimenter provides a large set of synthetic problems, a logging system, and a fast implementation. In this work, we refactor IOHexperimenter's logging system, in order to make it more extensible and modular. Using this new system, we implement a new logger, which aims at computing performance metrics of an algorithm across a benchmark. The logger computes the most generic view on an anytime stochastic heuristic performances, in the form of the Empirical Attainment Function (EAF). We also provide some common statistics on the EAF and its discrete counterpart, the Empirical Attainment Histogram. Our work has eventually been merged in the IOHexperimenter codebase.
翻译:为了能够进行大规模、景观意识的、按部就班的算法选择,基准平台软件是关键。 IOH 实验员提供了大量合成问题、伐木系统和快速实施。在这项工作中,我们重新构思了IOH实验员的伐木系统,以便使其更加可扩展和模块化。我们使用这个新系统,实施了一个新的记录仪,目的是计算一个跨越基准的算法的性能尺度。对随时的超常性能性能进行最通用的计算,其形式是“经验性耐久性功能 ” ( EAF) 。我们还提供了一些关于EAF及其离散对应方的通用统计数据,即“经验性耐久性直图 ” 。我们的工作最终被合并到 IOH 实验员代码库中。