Neural architecture search (NAS) has been studied extensively and has grown to become a research field with substantial impact. While classical single-objective NAS searches for the architecture with the best performance, multi-objective NAS considers multiple objectives that should be optimized simultaneously, e.g., minimizing resource usage along the validation error. Although considerable progress has been made in the field of multi-objective NAS, we argue that there is some discrepancy between the actual optimization problem of practical interest and the optimization problem that multi-objective NAS tries to solve. We resolve this discrepancy by formulating the multi-objective NAS problem as a quality diversity optimization (QDO) problem and introduce three quality diversity NAS optimizers (two of them belonging to the group of multifidelity optimizers), which search for high-performing yet diverse architectures that are optimal for application-specific niches, e.g., hardware constraints. By comparing these optimizers to their multi-objective counterparts, we demonstrate that quality diversity NAS in general outperforms multi-objective NAS with respect to quality of solutions and efficiency. We further show how applications and future NAS research can thrive on QDO.
翻译:虽然对神经结构搜索(NAS)进行了广泛研究,并发展成为具有重大影响的研究领域,虽然典型的单一目的NAS搜索建筑时表现最佳,但多目标NAS认为应同时优化多个目标,例如:在验证错误时尽量减少资源使用;虽然在多目标NAS领域取得了相当大的进展,但我们认为,在实际利益的实际优化问题与多目标NAS试图解决的优化问题之间存在着一定的差异;我们通过将多目标NAS问题作为质量多样性优化问题来解决这一差异,并引入三种质量多样性NAS优化器(其中两个属于多面性优化器组),以寻找适合具体应用点(例如硬件限制)的高性但多样化的建筑;通过将这些优化器与其多目标对应器进行比较,我们证明,在解决方案质量和效率方面,质量多样性NAS总体上优于多目标的NAS。我们进一步表明,应用和未来NAS研究如何能够在QDO上蓬勃发展。