Continual learning for segmentation has recently seen increasing interest. However, all previous works focus on narrow semantic segmentation and disregard panoptic segmentation, an important task with real-world impacts. %a In this paper, we present the first continual learning model capable of operating on both semantic and panoptic segmentation. Inspired by recent transformer approaches that consider segmentation as a mask-classification problem, we design CoMFormer. Our method carefully exploits the properties of transformer architectures to learn new classes over time. Specifically, we propose a novel adaptive distillation loss along with a mask-based pseudo-labeling technique to effectively prevent forgetting. To evaluate our approach, we introduce a novel continual panoptic segmentation benchmark on the challenging ADE20K dataset. Our CoMFormer outperforms all the existing baselines by forgetting less old classes but also learning more effectively new classes. In addition, we also report an extensive evaluation in the large-scale continual semantic segmentation scenario showing that CoMFormer also significantly outperforms state-of-the-art methods.
翻译:持续学习分化最近引起了越来越多的兴趣。然而,所有先前的工作都侧重于狭义的语义分解和忽略全光分解,这是具有真实世界影响的重要任务。%a 在本文中,我们展示了第一个能够同时运行语义和全光分解的连续学习模式。受最近将分解视为遮罩分解问题的变压器方法的启发,我们设计了ComFormer。我们的方法仔细利用变压器结构的特性来学习新班级。具体地说,我们提出了一种新的适应性蒸馏损失,同时提出了一种基于面罩的假标签技术,以有效防止遗忘。为了评估我们的方法,我们引入了具有挑战性的ADE20K数据集的新的全光分解基准。我们的COMFormer将所有现有基线都比不上现有的基准,因为它忘记了较少的旧班级,但也学习了更有效的新班级。此外,我们还报告了大规模持续分解假设中的广泛评价,表明CMFormer也明显地超越了艺术状态方法。