One common failure mode of Neural Radiance Field (NeRF) models is fitting incorrect geometries when given an insufficient number of input views. We propose DS-NeRF (Depth-supervised Neural Radiance Fields), a loss for learning neural radiance fields that takes advantage of readily-available depth supervision. Our key insight is that sparse depth supervision can be used to regularize the learned geometry, a crucial component for effectively rendering novel views using NeRF. We exploit the fact that current NeRF pipelines require images with known camera poses that are typically estimated by running structure-from-motion (SFM). Crucially, SFM also produces sparse 3D points that can be used as ``free" depth supervision during training: we simply add a loss to ensure that depth rendered along rays that intersect these 3D points is close to the observed depth. We find that DS-NeRF can render more accurate images given fewer training views while training 2-6x faster. With only two training views on real-world images, DS-NeRF significantly outperforms NeRF as well as other sparse-view variants. We show that our loss is compatible with these NeRF models, demonstrating that depth is a cheap and easily digestible supervisory signal. Finally, we show that DS-NeRF supports other types of depth supervision such as scanned depth sensors and RGBD reconstruction outputs.
翻译:神经辐射场( NERF) 模型的一个常见失败模式是当输入视图数量不足时,适合不正确的地貌。 我们建议DS- NERF (Depth- 由Depth监督的神经辐射场), 学习神经光亮场的损失, 利用随时可得的深度监督。 我们的关键见解是, 可以利用稀薄的深度监督来规范所学的几何学, 这是使用 NERF 有效提供新观点的关键组成部分。 我们利用以下事实,即当前的 NERF 管道需要已知的图像,这些图像通常通过运行结构(SFM)来估计。 很显然, SFM 也生成了稀疏的3D点, 可以在培训期间用作“ 自由” 深度监督: 我们只是增加一个损失, 以确保这些三维点相交集点的射线的深度接近观察深度。 我们发现 DS- NERF 在培训2-6x速度加快的同时可以提供更准确的图像。 我们只要对现实世界图像有两种培训观点, DS- NERF 明显超越了NRF, 以及其它低度深度的深度监管。 我们展示了其他的深度, 我们展示了这些深度的频率和低深度, 我们展示的模型展示的模型展示了我们展示了其他的模型。