Neural radiance fields (NeRFs) produce state-of-the-art view synthesis results. However, they are slow to render, requiring hundreds of network evaluations per pixel to approximate a volume rendering integral. Baking NeRFs into explicit data structures enables efficient rendering, but results in a large increase in memory footprint and, in many cases, a quality reduction. In this paper, we propose a novel neural light field representation that, in contrast, is compact and directly predicts integrated radiance along rays. Our method supports rendering with a single network evaluation per pixel for small baseline light field datasets and can also be applied to larger baselines with only a few evaluations per pixel. At the core of our approach is a ray-space embedding network that maps the 4D ray-space manifold into an intermediate, interpolable latent space. Our method achieves state-of-the-art quality on dense forward-facing datasets such as the Stanford Light Field dataset. In addition, for forward-facing scenes with sparser inputs we achieve results that are competitive with NeRF-based approaches in terms of quality while providing a better speed/quality/memory trade-off with far fewer network evaluations.
翻译:神经光亮场( NeRFs) 产生最新的视觉合成结果 。 但是, 速度缓慢, 需要每像素数百次网络评价, 以接近一个整体体积。 将 NERFs 引入明确的数据结构可以高效地生成, 但导致记忆足迹的大幅增长, 并在许多情况下, 质量下降。 在本文中, 我们提出一个新的神经光光场代表, 相对而言, 是紧凑的, 直接预测射线上的综合光谱。 我们的方法支持以单一网络评价每像素来提供小型基线光场数据集的小型基线光线像素, 也可以应用于较大的基线, 每像素只有很少的评价。 我们的方法的核心是一个射线空间嵌入网络, 将4D射线空间组合映射成中间的、 可间隙隐蔽空间。 我们的方法在像斯坦福光场数据集这样的密集远远方数据集上达到了最先进的质量。 此外, 对于带有稀薄投入的远方图像, 我们取得的结果, 与以低质量的网络相比具有竞争力。