Much recent progress has been made in reconstructing the 3D shape of an object from an image of it, i.e. single view 3D reconstruction. However, it has been suggested that current methods simply adopt a "nearest-neighbor" strategy, instead of genuinely understanding the shape behind the input image. In this paper, we rigorously show that for many state of the art methods, this issue manifests as (1) inconsistencies between coarse reconstructions and input images, and (2) inability to generalize across domains. We thus propose REFINE, a postprocessing mesh refinement step that can be easily integrated into the pipeline of any black-box method in the literature. At test time, REFINE optimizes a network per mesh instance, to encourage consistency between the mesh and the given object view. This, along with a novel combination of regularizing losses, reduces the domain gap and achieves state of the art performance. We believe that this novel paradigm is an important step towards robust, accurate reconstructions, remaining relevant as new reconstruction networks are introduced.
翻译:最近,在从一个物体的图像中重建一个物体的 3D 形状方面取得了很大进展, 即单一视图 3D 重建。 但是, 有人建议, 目前的方法只是采用“ 近邻” 战略, 而不是真正理解输入图像背后的形状。 在本文中, 我们严格地表明, 对于许多先进的方法来说, 这个问题表现为:(1) 粗糙的重建与输入图像之间不一致, 以及(2) 无法在跨域上一概而论。 因此, 我们提议了 REFINE, 后处理网形改进步骤, 可以很容易地融入任何黑盒方法的管道中。 在试验时, REFINE 优化每个网状的网络, 以鼓励网状和给定对象视图之间的一致。 这加上将损失正规化的新组合, 缩小了域间差距, 并实现艺术表现的状态。 我们认为, 这个新模式是走向强大、 准确的重建的重要一步, 随着新的重建网络被引入, 仍然具有相关性 。