A continual learning (CL) algorithm learns from a non-stationary data stream. The non-stationarity is modeled by some schedule that determines how data is presented over time. Most current methods make strong assumptions on the schedule and have unpredictable performance when such requirements are not met. A key challenge in CL is thus to design methods robust against arbitrary schedules over the same underlying data, since in real-world scenarios schedules are often unknown and dynamic. In this work, we introduce the notion of schedule-robustness for CL and a novel approach satisfying this desirable property in the challenging online class-incremental setting. We also present a new perspective on CL, as the process of learning a schedule-robust predictor, followed by adapting the predictor using only replay data. Empirically, we demonstrate that our approach outperforms existing methods on CL benchmarks for image classification by a large margin.
翻译:持续学习的算法从非静止数据流中学习。 非静止的算法以某些时间表为模型,这些时间表决定了数据如何随时间推移提供。大多数现行方法都对时间表作出强烈的假设,在无法满足这些要求时业绩无法预测。因此,CL的关键挑战是设计一些方法,以对付同一基本数据上的任意时间表,因为在现实世界情景中,时间表往往不为人知,而且动态不一。在这项工作中,我们引入了CL时间表-罗盘概念,并在具有挑战性的在线分类环境里采用了一种满足这一理想属性的新办法。我们还介绍了CL的新视角,作为学习一个时间表-罗盘预测器的过程,然后仅用重放数据调整预测器。我们很生动地表明,我们的方法超越了CL图像分类的现有方法,以大幅度进行分类。