Incremental semantic segmentation(ISS) is an emerging task where old model is updated by incrementally adding new classes. At present, methods based on convolutional neural networks are dominant in ISS. However, studies have shown that such methods have difficulty in learning new tasks while maintaining good performance on old ones (catastrophic forgetting). In contrast, a Transformer based method has a natural advantage in curbing catastrophic forgetting due to its ability to model both long-term and short-term tasks. In this work, we explore the reasons why Transformer based architecture are more suitable for ISS, and accordingly propose propose TISS, a Transformer based method for Incremental Semantic Segmentation. In addition, to better alleviate catastrophic forgetting while preserving transferability on ISS, we introduce two patch-wise contrastive losses to imitate similar features and enhance feature diversity respectively, which can further improve the performance of TISS. Under extensive experimental settings with Pascal-VOC 2012 and ADE20K datasets, our method significantly outperforms state-of-the-art incremental semantic segmentation methods.
翻译:递增语义分解( ISS) 是一个新兴任务, 旧模型通过渐进式添加新类别更新。 目前, 基于进化神经网络的方法在国际空间站中占主导地位。 但是, 研究表明, 这种方法在学习新任务的同时难以在老任务上保持良好的表现( 灾难性的遗忘 ) 。 相反, 以变异器为基础的方法在遏制灾难性的遗忘方面具有自然优势, 因为它有能力模拟长期和短期任务。 在这项工作中, 我们探讨了基于变异器的建筑更适合国际空间站的原因, 并相应提出了基于变异器的递增语义分解法。 此外, 为了更好地减轻灾难性的遗忘, 同时保护国际空间站的可转移性, 我们引入了两种偏差的对比性损失, 分别模仿相似的特征, 并增强特征的多样性, 这可以进一步提高 TISS的性能。 在广泛实验环境中, Pascal- VOC 2012 和 ADE20K 数据集下, 我们的方法大大优于最新式的递增语义分解方法。