For the semantic segmentation of images, state-of-the-art deep neural networks (DNNs) achieve high segmentation accuracy if that task is restricted to a closed set of classes. However, as of now DNNs have limited ability to operate in an open world, where they are tasked to identify pixels belonging to unknown objects and eventually to learn novel classes, incrementally. Humans have the capability to say: I don't know what that is, but I've already seen something like that. Therefore, it is desirable to perform such an incremental learning task in an unsupervised fashion. We introduce a method where unknown objects are clustered based on visual similarity. Those clusters are utilized to define new classes and serve as training data for unsupervised incremental learning. More precisely, the connected components of a predicted semantic segmentation are assessed by a segmentation quality estimate. connected components with a low estimated prediction quality are candidates for a subsequent clustering. Additionally, the component-wise quality assessment allows for obtaining predicted segmentation masks for the image regions potentially containing unknown objects. The respective pixels of such masks are pseudo-labeled and afterwards used for re-training the DNN, i.e., without the use of ground truth generated by humans. In our experiments we demonstrate that, without access to ground truth and even with few data, a DNN's class space can be extended by a novel class, achieving considerable segmentation accuracy.
翻译:对于图像的语义分解, 最先进的深神经网络( DNNS) 如果任务仅限于一组封闭的类, 则会达到高分解精度。 但是, 从现在起, DNNS 在开放的世界上的操作能力有限, 他们的任务是识别属于未知对象的像素, 并最终学习新类。 人类有能力说: 我不知道这是什么, 但我已经看到了类似的东西。 因此, 最好以一种不受监督的方式执行这样一个递增的学习任务 。 我们引入了一种方法, 将未知对象按视觉相似性分组。 这些分组被用来定义新类, 并用作不受监控的渐进学习的培训数据。 更准确地说, 预测的语义分解的连接部分通过分解质量估计来评估。 连接到预测质量低的组件是下一个组合的候选成分。 此外, 组合质量评估允许为可能包含未知对象的图像区域获取预测的分解面面面罩。 不同的口号是, 各自的代号是, 甚至由不使用虚拟标签的磁段进行实地测试, i 。