The task of continual learning requires careful design of algorithms that can tackle catastrophic forgetting. However, the noisy label, which is inevitable in a real-world scenario, seems to exacerbate the situation. While very few studies have addressed the issue of continual learning under noisy labels, long training time and complicated training schemes limit their applications in most cases. In contrast, we propose a simple purification technique to effectively cleanse the online data stream that is both cost-effective and more accurate. After purification, we perform fine-tuning in a semi-supervised fashion that ensures the participation of all available samples. Training in this fashion helps us learn a better representation that results in state-of-the-art (SOTA) performance. Through extensive experimentation on 3 benchmark datasets, MNIST, CIFAR10 and CIFAR100, we show the effectiveness of our proposed approach. We achieve a 24.8% performance gain for CIFAR10 with 20% noise over previous SOTA methods. Our code is publicly available.
翻译:持续学习的任务要求仔细设计能够解决灾难性遗忘问题的算法。然而,在现实世界中不可避免的吵闹标签似乎使情况恶化。尽管很少有研究涉及在吵闹标签下持续学习的问题,但长时间的培训时间和复杂的培训计划在多数情况下限制了它们的应用。相反,我们建议一种简单的净化技术,以有效地净化在线数据流,这种技术既具有成本效益,又更准确。净化后,我们以半监督的方式进行微调,确保所有现有样品的参与。这种方式的培训有助于我们学习更好的代表性,从而取得最先进的(SOTA)性能。通过对3个基准数据集(MMIST、CIFAR10和CIFAR100)的广泛试验,我们展示了我们拟议方法的有效性。我们为CIFAR10实现了24.8%的绩效收益,比以前的SOTA方法增加了20%的噪音。我们的代码是公开的。