Unsupervised domain adaptation (UDA) for semantic segmentation is a promising task freeing people from heavy annotation work. However, domain discrepancies in low-level image statistics and high-level contexts compromise the segmentation performance over the target domain. A key idea to tackle this problem is to perform both image-level and feature-level adaptation jointly. Unfortunately, there is a lack of such unified approaches for UDA tasks in the existing literature. This paper proposes a novel UDA pipeline for semantic segmentation that unifies image-level and feature-level adaptation. Concretely, for image-level domain shifts, we propose a global photometric alignment module and a global texture alignment module that align images in the source and target domains in terms of image-level properties. For feature-level domain shifts, we perform global manifold alignment by projecting pixel features from both domains onto the feature manifold of the source domain; and we further regularize category centers in the source domain through a category-oriented triplet loss and perform target domain consistency regularization over augmented target domain images. Experimental results demonstrate that our pipeline significantly outperforms previous methods. In the commonly tested GTA5$\rightarrow$Cityscapes task, our proposed method using Deeplab V3+ as the backbone surpasses previous SOTA by 8%, achieving 58.2% in mIoU.
翻译:用于语义分解的不受监督域适应(UDA)是一个很有希望的任务,可以使人们摆脱大量语义化工作。然而,低层次图像统计和高层次背景的域差异会影响目标域的分解性能。解决这一问题的一个关键想法是同时进行图像级别和地平级的调整。遗憾的是,现有文献中缺乏对语义分解任务的这种统一方法。本文件提议了一个新的 UDA 语义分解管道,以统一图像级别和地平级的调整。具体地说,对于图像级别域转移,我们建议了一个全球光度调整模块和一个全球纹理调整模块,在图像级别属性方面对源和目标域的图像进行对齐。对于地平级域变化,我们通过将两个域的像素特性投射到源域域的特征元体;我们进一步规范源域的分类中心,通过以类别为导向的三重损失,对目标域图域图进行目标域统一。实验结果显示,我们的管道价格大大超过我们先前采用的方法。在共同测试的GTA5-RMxxx中,通过共同测试的G-TA_Bxxxxxxx8xxxxxxxxxx8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx58xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx