The Novelty Search (NS) algorithm was proposed more than a decade ago. However, the mechanisms behind its empirical success are still not well formalized/understood. This short note focuses on the effects of the archive on exploration. Experimental evidence from a few application domains suggests that archive-based NS performs in general better than when Novelty is solely computed with respect to the population. An argument that is often encountered in the literature is that the archive prevents exploration from backtracking or cycling, i.e. from revisiting previously encountered areas in the behavior space. We argue that this is not a complete or accurate explanation as backtracking - beside often being desirable - can actually be enabled by the archive. Through low-dimensional/analytical examples, we show that a key effect of the archive is that it counterbalances the exploration biases that result, among other factors, from the use of inadequate behavior metrics and the non-linearities of the behavior mapping. Our observations seem to hint that attributing a more active role to the archive in sampling can be beneficial.
翻译:10多年前就提出了新奇搜索算法。 但是,其成功经验背后的机制还没有很好地正式化/理解。 本简短说明侧重于档案对勘探的影响。 几个应用领域的实验证据表明,一般而言,基于档案的NS的表现比仅仅根据人口计算新奇时要好。 文献中经常遇到的一个论点是,档案阻止勘探的回溯或循环,即无法重新审视先前在行为空间中遇到的领域。 我们认为,这并非一个完整或准确的解释,因为档案实际上可以进行回溯追踪,而不是通常可取的回溯。 我们通过低维/分析实例表明,档案的一个关键效果是,它抵消了由于行为指标不足和行为绘图的非线性而导致的探索偏差。 我们的观察似乎暗示,在取样中赋予档案以更积极的作用可能是有益的。