3D shape reconstruction from a single image has been a long-standing problem in computer vision. The problem is ill-posed and highly challenging due to the information loss and occlusion that occurred during the imagery capture. In contrast to previous methods that learn holistic shape priors, we propose a method to learn spatial pattern priors for inferring the invisible regions of the underlying shape, wherein each 3D sample in the implicit shape representation is associated with a set of points generated by hand-crafted 3D mappings, along with their local image features. The proposed spatial pattern is significantly more informative and has distinctive descriptions on both visible and occluded locations. Most importantly, the key to our work is the ubiquitousness of the spatial patterns across shapes, which enables reasoning invisible parts of the underlying objects and thus greatly mitigates the occlusion issue. We devise a neural network that integrates spatial pattern representations and demonstrate the superiority of the proposed method on widely used metrics.
翻译:从单一图像重建 3D 形状一直是计算机视觉中长期存在的一个问题。 由于在图像捕捉过程中出现的信息丢失和隐蔽性, 问题不正确且具有高度挑战性。 与以往学习整体形状前置法的方法不同, 我们建议了一种方法来学习空间模式前置法, 以推断隐形形状的隐形区域, 其中隐含形状的每个3D样本都与手工制作的 3D 绘图产生的一组点及其本地图像特征相关联。 拟议的空间模式信息量大得多, 并且对可见和隐蔽的位置都有独特的描述。 最重要的是, 我们工作的关键是跨形状的空间模式的无处可见性, 这使得能够推理基本物体的隐形部分, 从而大大减轻隐含性的问题。 我们设计了一个神经网络, 将空间图案的显示与广泛使用的测量方法的优越性进行整合。