Recently, inspired by DETR variants, query-based end-to-end instance segmentation (QEIS) methods have outperformed CNN-based models on large-scale datasets. Yet they would lose efficacy when only a small amount of training data is available since it's hard for the crucial queries/kernels to learn localization and shape priors. To this end, this work offers a novel unsupervised pre-training solution for low-data regimes. Inspired by the recent success of the Prompting technique, we introduce a new pre-training method that boosts QEIS models by giving Saliency Prompt for queries/kernels. Our method contains three parts: 1) Saliency Masks Proposal is responsible for generating pseudo masks from unlabeled images based on the saliency mechanism. 2) Prompt-Kernel Matching transfers pseudo masks into prompts and injects the corresponding localization and shape priors to the best-matched kernels. 3) Kernel Supervision is applied to supply supervision at the kernel level for robust learning. From a practical perspective, our pre-training method helps QEIS models achieve a similar convergence speed and comparable performance with CNN-based models in low-data regimes. Experimental results show that our method significantly boosts several QEIS models on three datasets. Code will be made available.
翻译:最近,在DETR变量的启发下,基于查询的端到端实例分解方法(QEIS)在大型数据集方面优于CNN的模型,但当只有少量的培训数据可用时,这些方法就会失去效力,因为关键查询/核心很难学习本地化和前置形状。为此,这项工作为低数据制度提供了一个全新的、不受监督的预培训解决方案。在 " 提示技术 " 的最近成功激励下,我们引入了新的培训前方法,通过对查询/核心给予 " 团结提示 " 来提升QEIS模式。我们的方法包括三个部分:1) " 团结面具建议 " 负责从基于突出机制的无标签图像中生成假面具。2) " 灵丹 " 匹配 " 将假面具传输到提示中,并引导相应的本地化和形状在最相配的内核系统之前。3) " 凯尔内 " 监督 " 将被用于在核心一级提供强化学习的监督。从实用角度出发,我们的培训前方法包括了三个部分:1) " 团结面具面具 " 建议 " 负责从突出的模型中生成假的模拟推进模型。