We tackle the problem of unsupervised synthetic-to-realistic domain adaptation for single image depth estimation. An essential building block of single image depth estimation is an encoder-decoder task network that takes RGB images as input and produces depth maps as output. In this paper, we propose a novel training strategy to force the task network to learn domain invariant representations in a self-supervised manner. Specifically, we extend self-supervised learning from traditional representation learning, which works on images from a single domain, to domain invariant representation learning, which works on images from two different domains by utilizing an image-to-image translation network. Firstly, we use our bidirectional image-to-image translation network to transfer domain-specific styles between synthetic and real domains. This style transfer operation allows us to obtain similar images from the different domains. Secondly, we jointly train our task network and Siamese network with the same images from the different domains to obtain domain invariance for the task network. Finally, we fine-tune the task network using labeled synthetic and unlabeled real-world data. Our training strategy yields improved generalization capability in the real-world domain. We carry out an extensive evaluation on two popular datasets for depth estimation, KITTI and Make3D. The results demonstrate that our proposed method outperforms the state-of-the-art both qualitatively and quantitatively. The source code and model weights will be made available.
翻译:我们处理未经监督的合成到现实领域适应单一图像深度估计的问题。 单一图像深度估计的一个基本组成部分是将 RGB 图像作为输入输入并制作深度地图作为输出的编码器- 解码器任务网络。 在本文中,我们提出了一个新的培训战略, 迫使任务网络以自我监督的方式学习域差异表示。 具体地说, 我们从传统代表学习中进行自我监督的学习, 从一个单一域的图像上进行工作, 到域内差异表示式学习, 通过利用图像到模拟翻译网络对两个不同域的图像进行工作。 首先, 我们使用双向图像到映像翻译网络在合成领域和真实领域之间传输特定域的样式。 我们的这种风格转移操作使我们得以从不同领域获得相似的图像。 其次, 我们联合培训我们的任务网络和暹米网络, 从不同域的模型中获取相同的图像, 到任务网络的域域域的不变化。 最后, 我们将使用标签合成和未标定真实世界版数翻译网络对任务源网络进行微调。 我们的培训战略将产生一个大范围的域域图, 将显示我们现有的域域图的深度, 我们的域图中的数据系统将展示。 将改进了我们现有的域图, 将改进了我们现有的域法 。