Although deep learning has made significant progress on fixed large-scale datasets, it typically encounters challenges regarding improperly detecting unknown/unseen classes in the open-world scenario, over-parametrized, and overfitting small samples. Since biological systems can overcome the above difficulties very well, individuals inherit an innate gene from collective creatures that have evolved over hundreds of millions of years and then learn new skills through few examples. Inspired by this, we propose a practical collective-individual paradigm where an evolution (expandable) network is trained on sequential tasks and then recognize unknown classes in real-world. Moreover, the learngene, i.e., the gene for learning initialization rules of the target model, is proposed to inherit the meta-knowledge from the collective model and reconstruct a lightweight individual model on the target task. Particularly, a novel criterion is proposed to discover learngene in the collective model, according to the gradient information. Finally, the individual model is trained only with few samples on the target learning tasks. We demonstrate the effectiveness of our approach in an extensive empirical study and theoretical analysis.
翻译:虽然在固定的大型数据集方面已经取得了重大进步,但通常在不适当地发现开放世界情景中的未知/未见类别、过度平衡和过度配置小样本方面会遇到挑战;由于生物系统可以很好地克服上述困难,个人继承了数亿年发展起来的集体生物的遗传基因,然后通过几个例子学习新的技能。受此启发,我们提出了一个实用的集体-个人模式,在这种模式下,对进化(可扩展)网络进行相继任务培训,然后在现实世界中识别未知类别。此外,还提议学习基因,即学习目标模型初始化规则的基因,从集体模型中继承元知识,并重建目标任务方面的轻量体型个人模型。特别是,根据梯度信息,建议采用新的标准在集体模型中发现学习基因。最后,对单个模型的培训仅以很少的样本进行目标学习任务。我们在广泛的实验研究和理论分析中展示了我们的方法的有效性。