Existing depth completion methods are often targeted at a specific sparse depth type, and generalize poorly across task domains. We present a method to complete sparse/semi-dense, noisy, and potentially low-resolution depth maps obtained by various range sensors, including those in modern mobile phones, or by multi-view reconstruction algorithms. Our method leverages a data driven prior in the form of a single image depth prediction network trained on large-scale datasets, the output of which is used as an input to our model. We propose an effective training scheme where we simulate various sparsity patterns in typical task domains. In addition, we design two new benchmarks to evaluate the generalizability and the robustness of depth completion methods. Our simple method shows superior cross-domain generalization ability against state-of-the-art depth completion methods, introducing a practical solution to high quality depth capture on a mobile device. Code is available at: https://github.com/YvanYin/FillDepth.
翻译:现有的深度完成方法往往针对特定稀疏的深度类型,并且对各任务领域进行概括化。我们提出了一个方法,以完成由各种射程传感器(包括现代移动电话中的传感器)或多视重建算法获得的稀疏/半敏度、噪音和潜在的低分辨率深度地图。我们的方法利用了以前以单一图像深度预测网络为驱动的数据,这种数据是经过大规模数据集培训的,其产出被用作对模型的投入。我们提出了一个有效的培训计划,用以模拟典型任务领域的各种宽度模式。此外,我们设计了两个新的基准,以评价深度完成方法的一般性和稳健性。我们的简单方法显示相对于最先进的深度完成方法的高级跨部通用能力,为在移动设备上高品质深度捕捉取引入实用解决方案。代码见于:https://github.com/YvanYin/FillDepth。