In this paper, we investigate a new optimization framework for multi-view 3D shape reconstructions. Recent differentiable rendering approaches have provided breakthrough performances with implicit shape representations though they can still lack precision in the estimated geometries. On the other hand multi-view stereo methods can yield pixel wise geometric accuracy with local depth predictions along viewing rays. Our approach bridges the gap between the two strategies with a novel volumetric shape representation that is implicit but parameterized with pixel depths to better materialize the shape surface with consistent signed distances along viewing rays. The approach retains pixel-accuracy while benefiting from volumetric integration in the optimization. To this aim, depths are optimized by evaluating, at each 3D location within the volumetric discretization, the agreement between the depth prediction consistency and the photometric consistency for the corresponding pixels. The optimization is agnostic to the associated photo-consistency term which can vary from a median-based baseline to more elaborate criteria learned functions. Our experiments demonstrate the benefit of the volumetric integration with depth predictions. They also show that our approach outperforms existing approaches over standard 3D benchmarks with better geometry estimations.
翻译:在本文中, 我们为多视图 3D 形状重建探索一个新的优化框架。 最近不同的转换方法提供了隐含形状表示的突破性表现, 虽然它们仍然可能缺乏估计的几何的精确度。 另一方面, 多视图立体法可以产生像素的智能几何精确度, 并随观察光线进行局部深度预测。 我们的方法可以缩小两种战略之间的差距, 其新型的体积形状表示法是隐含的, 但以像素深度为参数, 以更好地实现形状表面, 与观察射线一致的距离。 这种方法保留了像素的精度, 并受益于优化的体积整合。 为此, 多视图立体法方法通过在体积分解的每个地点对深度预测一致性和相应像素光度一致性之间的协议进行评估, 来优化深度的深度。 优化对于相关的光一致性术语来说, 从中位基基线到更精细的标准学习功能不等。 我们的实验表明, 体积整合的好处与深度预测的好处。 它们还表明, 我们的方法超越了现有标准3的测量方法。</s>