By supervising camera rays between a scene and multi-view image planes, NeRF reconstructs a neural scene representation for the task of novel view synthesis. On the other hand, shadow rays between the light source and the scene have yet to be considered. Therefore, we propose a novel shadow ray supervision scheme that optimizes both the samples along the ray and the ray location. By supervising shadow rays, we successfully reconstruct a neural SDF of the scene from single-view images under multiple lighting conditions. Given single-view binary shadows, we train a neural network to reconstruct a complete scene not limited by the camera's line of sight. By further modeling the correlation between the image colors and the shadow rays, our technique can also be effectively extended to RGB inputs. We compare our method with previous works on challenging tasks of shape reconstruction from single-view binary shadow or RGB images and observe significant improvements. The code and data are available at https://github.com/gerwang/ShadowNeuS.
翻译:通过对场景和多视角图像平面之间的相机光线进行监督,NeRF 重建了神经场景表示,用于新视角合成任务。另一方面,尚未考虑光源和场景之间的阴影光线。因此,我们提出了一种新的阴影光线监督方案,可以同时优化沿光线的样本和光线位置。通过监督阴影光线,我们成功地从单视角图像在多种照明条件下重建了场景的神经SDF。给定单视角二进制阴影,我们训练神经网络以重建一个不受摄像机视线限制的完整场景。通过进一步建模图像颜色和阴影光线之间的相关性,我们的技术也可以有效扩展到 RGB 输入。我们在挑战性的从单视角二进制阴影或 RGB 图像重建形状的任务上,与以前的工作进行了比较,观察到了显著的改进。代码和数据可在 https://github.com/gerwang/ShadowNeuS 找到。