Under good conditions, Neural Radiance Fields (NeRFs) have shown impressive results on novel view synthesis tasks. NeRFs learn a scene's color and density fields by minimizing the photometric discrepancy between training views and differentiable renders of the scene. Once trained from a sufficient set of views, NeRFs can generate novel views from arbitrary camera positions. However, the scene geometry and color fields are severely under-constrained, which can lead to artifacts, especially when trained with few input views. To alleviate this problem we learn a prior over scene geometry and color, using a denoising diffusion model (DDM). Our DDM is trained on RGBD patches of the synthetic Hypersim dataset and can be used to predict the gradient of the logarithm of a joint probability distribution of color and depth patches. We show that, during NeRF training, these gradients of logarithms of RGBD patch priors serve to regularize geometry and color for a scene. During NeRF training, random RGBD patches are rendered and the estimated gradients of the log-likelihood are backpropagated to the color and density fields. Evaluations on LLFF, the most relevant dataset, show that our learned prior achieves improved quality in the reconstructed geometry and improved generalization to novel views. Evaluations on DTU show improved reconstruction quality among NeRF methods.
翻译:在良好条件下,神经辐射场(Neoral Radiance Fields)在新视觉合成任务上取得了令人印象深刻的成果。 NeRFs通过最大限度地缩小培训视图和可变场面之间的光度差异来学习场景的颜色和密度域。一旦经过足够多的视图培训,NeRFs就可以从任意的相机位置产生新的观点。然而,场景几何和彩色域严重缺乏控制,这可能导致人工制品,特别是当经过少量投入观点的培训时。为了缓解这一问题,我们学习了先于场景的几何和颜色,使用了分辨扩散模型(DDM)。我们DDDM学会了场景的颜色和密度。我们DDDD在合成超正像数据集的RGBD补丁补丁方面受过培训,可以用来预测颜色和深度联合分布的概率分布的对数的梯度。我们显示,在NRFS培训期间,这些RD的对数梯度梯度梯度梯度梯度的梯度有助于对一个场景色和颜色进行规范化。在NRFS Realidestration上随机的对准和最接近的对地平地的对地的对地表进行重新显示前的颜色和对地的对地的对地表的对地表,对地的对地的对地的对地表,对地表的对地表的对地表显示,对地的对地的对地表,对地的对地的对地的改进后显示的对地的对地的对地的对地表的对地表的对地表的对地表的重新显示到对地表,对地的重新显示到对地表,对地的重新显示到对地的对地的对地表,对地表,对地的重新显示到对地的对地的对地的对地的对地的重新显示到对地的对地的对地的对地的对地的重新显示到对地的重新显示到对地的重新显示到对地的对地。