Novelty Detection methods identify samples that are not representative of a model's training set thereby flagging misleading predictions and bringing a greater flexibility and transparency at deployment time. However, research in this area has only considered Novelty Detection in the offline setting. Recently, there has been a growing realization in the computer vision community that applications demand a more flexible framework - Continual Learning - where new batches of data representing new domains, new classes or new tasks become available at different points in time. In this setting, Novelty Detection becomes more important, interesting and challenging. This work identifies the crucial link between the two problems and investigates the Novelty Detection problem under the Continual Learning setting. We formulate the Continual Novelty Detection problem and present a benchmark, where we compare several Novelty Detection methods under different Continual Learning settings. We show that Continual Learning affects the behaviour of novelty detection algorithms, while novelty detection can pinpoint insights in the behaviour of a continual learner. We further propose baselines and discuss possible research directions. We believe that the coupling of the two problems is a promising direction to bring vision models into practice.
翻译:新颖探索方法确定样本,并不代表模型培训,从而在部署时显示误导性预测,并带来更大的灵活性和透明度。然而,这一领域的研究仅考虑离线环境中的新颖探测。最近,计算机视觉界日益认识到,应用要求一个更灵活的框架----持续学习----即在不同时间点提供代表新领域、新类别或新任务的新数据,在这一背景下,新颖探测变得更加重要、有趣和具有挑战性。这项工作确定了这两个问题之间的关键联系,并调查了持续学习环境中的新颖探测问题。我们制定了持续新颖探测问题,并提出了一个基准,用以比较不同持续学习环境中的若干新颖探测方法。我们表明,持续学习会影响新颖探测算法的行为,而新颖的探测可以发现不断学习者的行为中的洞察力。我们进一步提出基线,并讨论可能的研究方向。我们认为,将这两个问题结合起来是将愿景模型付诸实践的一个充满希望的方向。