Few-shot class incremental learning -- the problem of updating a trained classifier to discriminate among an expanded set of classes with limited labeled data -- is a key challenge for machine learning systems deployed in non-stationary environments. Existing approaches to the problem rely on complex model architectures and training procedures that are difficult to tune and re-use. In this paper, we present an extremely simple approach that enables the use of ordinary logistic regression classifiers for few-shot incremental learning. The key to this approach is a new family of subspace regularization schemes that encourage weight vectors for new classes to lie close to the subspace spanned by the weights of existing classes. When combined with pretrained convolutional feature extractors, logistic regression models trained with subspace regularization outperform specialized, state-of-the-art approaches to few-shot incremental image classification by up to 22% on the miniImageNet dataset. Because of its simplicity, subspace regularization can be straightforwardly extended to incorporate additional background information about the new classes (including class names and descriptions specified in natural language); these further improve accuracy by up to 2%. Our results show that simple geometric regularization of class representations offers an effective tool for continual learning.
翻译:少见的班级递增学习 -- -- 更新经过训练的分类师,以区分一组有有限标签的数据的扩大班级的问题 -- -- 是对部署在非静止环境中的机器学习系统的一个关键挑战。问题的现有方法依赖于难以调和和和再使用的复杂的模型结构和培训程序。在本文中,我们提出了一个非常简单的方法,使普通后勤回归分类师能够用于少见的递增学习。这一方法的关键在于一个新的子空间正规化计划组合,鼓励新的班级的重量矢量与现有班级重量所覆盖的子空间相近。当与预先训练的转动特征提取器相结合时,经过辅助空间正规化训练的后勤回归模型将超出微小图像网数据集中专门、最先进的微小图像分类方法。由于其简单化,子空间正规化可以直接扩展,以纳入关于新班级的额外背景资料(包括以自然语言指定的班级名称和描述);这些方法进一步提高准确性,达到2%。我们的结果显示,简单的班级结构正规化为持续学习提供了有效的工具。