In this paper, we propose a neural architecture search framework based on a similarity measure between some baseline tasks and a target task. We first define the notion of the task similarity based on the log-determinant of the Fisher Information matrix. Next, we compute the task similarity from each of the baseline tasks to the target task. By utilizing the relation between a target and a set of learned baseline tasks, the search space of architectures for the target task can be significantly reduced, making the discovery of the best candidates in the set of possible architectures tractable and efficient, in terms of GPU days. This method eliminates the requirement for training the networks from scratch for a given target task as well as introducing the bias in the initialization of the search space from the human domain.
翻译:在本文中,我们提出一个神经结构搜索框架,其依据是某些基线任务和目标任务之间的类似度度;我们首先根据渔业信息矩阵的对称定义界定任务相似性的概念;接下来,我们根据每个基线任务与目标任务之间的相似性来计算任务;通过利用目标与一组已学习的基线任务之间的关系,可以大大减少目标任务结构的搜索空间,使在一组可移动和高效的可能的架构中找到最佳候选人,即GPU日;这种方法消除了从零到零培训网络完成特定目标任务的要求,并引入了从人类领域开始搜索空间的偏差。