In this paper, we propose a neural architecture search framework based on a similarity measure between the baseline tasks and the incoming target task. We first define the notion of task similarity based on the log-determinant of the Fisher Information Matrices. Next, we compute the task similarity from each of the baseline tasks to the incoming target task. By utilizing the relation between a target and a set of learned baseline tasks, the search space of architectures for the incoming target task can be significantly reduced, making the discovery of the best candidates in the set of possible architectures tractable and efficient, in terms of GPU days. This method eliminates the requirement for training the networks from scratch for the incoming target task as well as introducing the bias in the initialization of the search space from the human domain. Experimental results with 8 classification tasks in MNIST and CIFAR-10 datasets illustrate the efficacy of our proposed approach and its competitiveness with other state-of-art methods in terms of the classification performance, the number of parameters, and the search time.
翻译:在本文件中,我们根据基线任务和即将到来的目标任务之间的类似度度,提出神经结构搜索框架;我们首先根据渔业信息矩阵的日志来界定任务相似性的概念;接着,我们根据每个基线任务和即将到来的目标任务来计算任务相似性的任务;通过利用一个目标与一套已学习的基线任务之间的关系,可以大大减少即将到来的目标任务的结构搜索空间,从而在GPU日中发现一套可移动和高效的可能建筑中的最佳候选人;这种方法消除了对即将到来的目标任务的网络进行从零到零培训的要求,并且从人类领域开始搜索空间时引入了偏见;实验结果,在MNIST和CIFAR-10数据集中的8个分类任务表明了我们拟议方法的效率及其在分类性能、参数数目和搜索时间方面与其他最先进方法的竞争力。