Quality-Diversity (QD) algorithms are designed to generate collections of high-performing solutions while maximizing their diversity in a given descriptor space. However, in the presence of unpredictable noise, the fitness and descriptor of the same solution can differ significantly from one evaluation to another, leading to uncertainty in the estimation of such values. Given the elitist nature of QD algorithms, they commonly end up with many degenerate solutions in such noisy settings. In this work, we introduce Archive Reproducibility Improvement Algorithm (ARIA); a plug-and-play approach that improves the reproducibility of the solutions present in an archive. We propose it as a separate optimization module, relying on natural evolution strategies, that can be executed on top of any QD algorithm. Our module mutates solutions to (1) optimize their probability of belonging to their niche, and (2) maximize their fitness. The performance of our method is evaluated on various tasks, including a classical optimization problem and two high-dimensional control tasks in simulated robotic environments. We show that our algorithm enhances the quality and descriptor space coverage of any given archive by at least 50%.
翻译:质量多样性 (Quality-Diversity) 算法旨在在给定描述符空间内生成高性能解集合的同时最大化它们的多样性。然而,在存在不可预测噪声的情况下,同一解的适应度和描述符值可能会在不同的评估中存在显著差异,导致对这些值的估计存在不确定性。鉴于QD算法的精英主义性质,它们通常会在这种嘈杂的环境中得到许多退化解。在本文中,我们引入了存档行为再现性改进算法(ARIA),这是一种可插拔的方法,旨在改进归档中存在的解的再现性。我们将其作为一个独立的优化模块提出,依赖于自然进化策略,可以在任何QD算法之上执行。我们的模块将解变异为:(1)优化其属于其生态位的概率,(2)最大化其适应度。我们评估了我们方法在各种任务中的性能,包括一个经典的优化问题和在模拟机器人环境中的两个高维控制任务。我们展示了我们的算法将给定档案的质量和描述符空间覆盖面至少提高了50%。