We propose a novel deep reinforcement learning-based approach for 3D object reconstruction from monocular images. Prior works that use mesh representations are template based. Thus, they are limited to the reconstruction of objects that have the same topology as the template. Methods that use volumetric grids as intermediate representations are computationally expensive, which limits their application in real-time scenarios. In this paper, we propose a novel end-to-end method that reconstructs 3D objects of arbitrary topology from a monocular image. It is composed of of (1) a Vertex Generation Network (VGN), which predicts the initial 3D locations of the object's vertices from an input RGB image, (2) a differentiable triangulation layer, which learns in a non-supervised manner, using a novel reinforcement learning algorithm, the best triangulation of the object's vertices, and finally, (3) a hierarchical mesh refinement network that uses graph convolutions to refine the initial mesh. Our key contribution is the learnable triangulation process, which recovers in an unsupervised manner the topology of the input shape. Our experiments on ShapeNet and Pix3D benchmarks show that the proposed method outperforms the state-of-the-art in terms of visual quality, reconstruction accuracy, and computational time.
翻译:我们用单体图像为三维天体重建建议一种新型的深层强化学习法。 先前使用网状图示的工程以模板为基础。 因此, 它们仅限于重建与模板具有相同地形的物体。 使用体积网格作为中间图示的方法在计算上非常昂贵, 从而限制了其在实时情景中的应用。 在本文中, 我们提出一种新的端到端方法, 用单体图像重建三维的任意地貌物体。 它由 (1) 一个 Vertex 生成网络( VGN) 组成, 它从输入 RGB 图像中预测对象的顶部最初 3D 位置, (2) 一个不同的三角层, 它以非超导的方式学习, 使用新型的强化学习算法, 最佳的物体顶部三角组合, 最后, 我们用图形相形图的精细度改进网络。 我们的主要贡献是可学习的三角图解进程, 它以不受到控制的方式恢复了 输入的图像形状的表层学, 3 以不受到监督的方式, 我们的图像模型模型的模型 显示的图像模型的精确度 的模型 。