The recent advances in continual (incremental or lifelong) learning have concentrated on the prevention of forgetting that can lead to catastrophic consequences, but there are two outstanding challenges that must be addressed. The first is the evaluation of the robustness of the proposed methods. The second is ensuring the security of learned tasks remains largely unexplored. This paper presents a comprehensive study of the susceptibility of the continually learned tasks (including both current and previously learned tasks) that are vulnerable to forgetting. Such vulnerability of tasks against adversarial attacks raises profound issues in data integrity and privacy. We consider the task incremental learning (Task-IL) scenario and explore three regularization-based experiments, three replay-based experiments, and one hybrid technique based on the reply and exemplar approach. We examine the robustness of these methods. In particular, we consider cases where we demonstrate that any class belonging to the current or previously learned tasks is prone to misclassification. Our observations highlight the potential limitations of existing Task-IL approaches. Our empirical study recommends that the research community consider the robustness of the proposed continual learning approaches and invest extensive efforts in mitigating catastrophic forgetting.
翻译:最近不断(高等或终身)学习的进展集中于防止忘却可能导致灾难性后果的忘却,但有两个有待解决的挑战。第一个是评价拟议方法的稳健性。第二个是确保所学到任务的安全性,基本上尚未探索。本文件全面研究容易被遗忘的不断学习任务(包括当前和以往所学到的任务)的易感性。这种在对抗性攻击面前的脆弱性提出了数据完整性和隐私方面的深刻问题。我们考虑了任务递增(Task-IL)设想,并探索了三个基于正规化的实验、三个基于重现的实验和一种基于答复和特例方法的混合技术。我们研究了这些方法的稳健性。我们特别审议了一些情况,即我们发现属于当前或以往所学到的任务的任何类别都容易被错误分类。我们的意见强调了现有任务-IL方法的潜在局限性。我们的经验研究建议研究界考虑拟议的持续学习方法的稳健性,并作出广泛的努力,以减轻灾难性的遗忘。