Weakly supervised segmentation requires assigning a label to every pixel based on training instances with partial annotations such as image-level tags, object bounding boxes, labeled points and scribbles. This task is challenging, as coarse annotations (tags, boxes) lack precise pixel localization whereas sparse annotations (points, scribbles) lack broad region coverage. Existing methods tackle these two types of weak supervision differently: Class activation maps are used to localize coarse labels and iteratively refine the segmentation model, whereas conditional random fields are used to propagate sparse labels to the entire image. We formulate weakly supervised segmentation as a semi-supervised metric learning problem, where pixels of the same (different) semantics need to be mapped to the same (distinctive) features. We propose 4 types of contrastive relationships between pixels and segments in the feature space, capturing low-level image similarity, semantic annotation, co-occurrence, and feature affinity They act as priors; the pixel-wise feature can be learned from training images with any partial annotations in a data-driven fashion. In particular, unlabeled pixels in training images participate not only in data-driven grouping within each image, but also in discriminative feature learning within and across images. We deliver a universal weakly supervised segmenter with significant gains on Pascal VOC and DensePose. Our code is publicly available at https://github.com/twke18/SPML.
翻译:微弱监管分解要求根据培训实例为每个像素设置标签, 包括部分说明, 如图像级别标签、 对象约束框、 标签点和刻字。 这项任务具有挑战性, 因为粗略说明( 标签、 框) 缺乏精确的像素本地化, 而稀少说明( 点、 细略) 缺乏广泛的区域覆盖。 现有方法以不同的方式处理这两类薄弱的监管: 类激活地图用于将粗略标签本地化, 并迭接地完善分解模式, 而有条件随机字段则用于向整个图像传播稀释标签。 我们将微弱监管的分解作为半超超高的衡量学习问题。 我们从培训图像中学习微弱的分解分解, 同一( 不同) 语系( 细微) 语系的分解( 细略), 我们提议在特性空间的像素和区块之间有4种对比关系, 获取低层次的图像相似性、 语系笔记、 共同随机化的字段作为前科; 我们可以从培训图像中学习微的分解的分解, 。 在每个图像中学习中, 我们的易图像中, 以部分中, 仅制的解的分解图解中学习中, 。