Reconstructing 3D shapes from planar cross-sections is a challenge inspired by downstream applications like medical imaging and geographic informatics. The input is an in/out indicator function fully defined on a sparse collection of planes in space, and the output is an interpolation of the indicator function to the entire volume. Previous works addressing this sparse and ill-posed problem either produce low quality results, or rely on additional priors such as target topology, appearance information, or input normal directions. In this paper, we present OReX, a method for 3D shape reconstruction from slices alone, featuring a Neural Field as the interpolation prior. A modest neural network is trained on the input planes to return an inside/outside estimate for a given 3D coordinate, yielding a powerful prior that induces smoothness and self-similarities. The main challenge for this approach is high-frequency details, as the neural prior is overly smoothing. To alleviate this, we offer an iterative estimation architecture and a hierarchical input sampling scheme that encourage coarse-to-fine training, allowing the training process to focus on high frequencies at later stages. In addition, we identify and analyze a ripple-like effect stemming from the mesh extraction step. We mitigate it by regularizing the spatial gradients of the indicator function around input in/out boundaries during network training, tackling the problem at the root. Through extensive qualitative and quantitative experimentation, we demonstrate our method is robust, accurate, and scales well with the size of the input. We report state-of-the-art results compared to previous approaches and recent potential solutions, and demonstrate the benefit of our individual contributions through analysis and ablation studies.
翻译:从平面截面中重构3D形状是一个具有挑战性的问题,受到医学影像和地理信息学等下游应用的启发。该输入是在空间中定义的稀疏平面集上完全定义的内/外指示函数,而输出是指标函数到整个体积的插值。解决这个稀疏和不适定问题的先前工作要么产生低质量的结果,要么依赖于其他先验,如目标拓扑、外观信息或输入法线方向。在本文中,我们提出了一种名为OReX的方法,用于仅使用切片进行3D形状重构,以神经场作为插值先验。一个适度的神经网络被训练用于在给定一个3D坐标时返回一个内/外估计值,从而产生一个强大的先验,诱导平滑和自相似性。这种方法面临的主要挑战是高频细节,因为神经先验过度平滑。为了缓解这个问题,我们提供了一个迭代估计架构和一个分层输入采样方案,鼓励从粗到细的训练,使训练过程能够在后期集中关注高频率。此外,我们识别和分析了从网格提取步骤引起的类纹状效应。我们通过在网络训练过程中规范输入内/外边界周围的指示函数的空间梯度来缓解这个问题,从根本上解决了这个问题。通过广泛的定性和定量实验,我们证明了我们的方法具有鲁棒性、精确性和与输入大小良好的可扩展性。我们报告了与以前的方法和最近的潜在解决方案相比的最新结果,并通过分析和消融研究证明了我们各自贡献的好处。