We address the problem of incremental semantic segmentation (ISS) recognizing novel object/stuff categories continually without forgetting previous ones that have been learned. The catastrophic forgetting problem is particularly severe in ISS, since pixel-level ground-truth labels are available only for the novel categories at training time. To address the problem, regularization-based methods exploit probability calibration techniques to learn semantic information from unlabeled pixels. While such techniques are effective, there is still a lack of theoretical understanding of them. Replay-based methods propose to memorize a small set of images for previous categories. They achieve state-of-the-art performance at the cost of large memory footprint. We propose in this paper a novel ISS method, dubbed ALIFE, that provides a better compromise between accuracy and efficiency. To this end, we first show an in-depth analysis on the calibration techniques to better understand the effects on ISS. Based on this, we then introduce an adaptive logit regularizer (ALI) that enables our model to better learn new categories, while retaining knowledge for previous ones. We also present a feature replay scheme that memorizes features, instead of images directly, in order to reduce memory requirements significantly. Since a feature extractor is changed continually, memorized features should also be updated at every incremental stage. To handle this, we introduce category-specific rotation matrices updating the features for each category separately. We demonstrate the effectiveness of our approach with extensive experiments on standard ISS benchmarks, and show that our method achieves a better trade-off in terms of accuracy and efficiency.
翻译:我们处理递增语义分解(ISS)的问题,在不忘以前已学过的东西的情况下,不断承认新对象/新类别。灾难性的忘却问题在国际空间站中特别严重,因为像素级地面真相标签只在培训时间为新类别提供。为解决这一问题,基于正规化的方法利用概率校准技术从未贴标签的像素中学习语义信息。虽然这些技术是有效的,但理论上仍缺乏对这些技术的理解。基于重放的方法提议对以往类别中的一小套图像进行记忆化。它们以大记忆足迹的代价实现最先进的性能。我们在此文件中建议一种新型的国际空间站方法,在准确性能和效率之间提供更好的妥协。为此,我们首先对校准技术进行深入分析,以更好地了解对国际空间站的影响。在此基础上,我们随后引入了适应性对逻辑校正校正(AI)方法,让我们的模型更好地学习新类别,同时保留对前几类的准确性知识。我们还在本文中提出了一种新的新的国际空间站状态方法,在每一阶段都显示一个不断更新的缩缩缩缩缩图。