In this work, we present a fully self-supervised framework for semantic segmentation(FS^4). A fully bootstrapped strategy for semantic segmentation, which saves efforts for the huge amount of annotation, is crucial for building customized models from end-to-end for open-world domains. This application is eagerly needed in realistic scenarios. Even though recent self-supervised semantic segmentation methods have gained great progress, these works however heavily depend on the fully-supervised pretrained model and make it impossible a fully self-supervised pipeline. To solve this problem, we proposed a bootstrapped training scheme for semantic segmentation, which fully leveraged the global semantic knowledge for self-supervision with our proposed PGG strategy and CAE module. In particular, we perform pixel clustering and assignments for segmentation supervision. Preventing it from clustering a mess, we proposed 1) a pyramid-global-guided (PGG) training strategy to supervise the learning with pyramid image/patch-level pseudo labels, which are generated by grouping the unsupervised features. The stable global and pyramid semantic pseudo labels can prevent the segmentation from learning too many clutter regions or degrading to one background region; 2) in addition, we proposed context-aware embedding (CAE) module to generate global feature embedding in view of its neighbors close both in space and appearance in a non-trivial way. We evaluate our method on the large-scale COCO-Stuff dataset and achieved 7.19 mIoU improvements on both things and stuff objects
翻译:在这项工作中,我们提出了一个完全自我监督的语义分解框架(FS+4 4 ) 。 一个完全自监督的语义分解框架(FS+4 4 ) 。 一个完全自监督的语义分解战略(SFS+4 ), 省去大量注解的努力, 这对于从终端到终端为开放世界域构建自定义模型至关重要。 这个应用在现实的情景中非常需要。 尽管最近自监督的语义分解方法取得了巨大进展, 但是这些工程在很大程度上取决于完全监督的预先培训模式, 使得它不可能成为一个完全自监督的自我监督的管道。 为了解决这个问题, 我们提出了一个语义分解的语义分解培训计划, 将全球语义知识充分用于自我监督的自我监督模式和 CAE 模块。 稳定的全球语系组合和任务组合, 我们提议使用金字塔- 全球指南(PGGG) 培训战略, 以监督以金字塔图/patch- stalation 类比标的学习, 通过对不透视透视的事物进行分组方式进行分组, 其不透透视, 将全球语言分解的语义分解的内地段段段段段段段段系生成, 产生。