Novel view synthesis is a long-standing problem that revolves around rendering frames of scenes from novel camera viewpoints. Volumetric approaches provide a solution for modeling occlusions through the explicit 3D representation of the camera frustum. Multi-plane Images (MPI) are volumetric methods that represent the scene using front-parallel planes at distinct depths but suffer from depth discretization leading to a 2.D scene representation. Another line of approach relies on implicit 3D scene representations. Neural Radiance Fields (NeRF) utilize neural networks for encapsulating the continuous 3D scene structure within the network weights achieving photorealistic synthesis results, however, methods are constrained to per-scene optimization settings which are inefficient in practice. Multi-plane Neural Radiance Fields (MINE) open the door for combining implicit and explicit scene representations. It enables continuous 3D scene representations, especially in the depth dimension, while utilizing the input image features to avoid per-scene optimization. The main drawback of the current literature work in this domain is being constrained to single-view input, limiting the synthesis ability to narrow viewpoint ranges. In this work, we thoroughly examine the performance, generalization, and efficiency of single-view multi-plane neural radiance fields. In addition, we propose a new multiplane NeRF architecture that accepts multiple views to improve the synthesis results and expand the viewing range. Features from the input source frames are effectively fused through a proposed attention-aware fusion module to highlight important information from different viewpoints. Experiments show the effectiveness of attention-based fusion and the promising outcomes of our proposed method when compared to multi-view NeRF and MPI techniques.
翻译:多平面图像(MPI)是一种量性方法,在不同的深度使用前方和侧面平面平面平面平面平面平面,但因深度离散而导致2.D场面展示。另一行方法依赖于隐含的 3D 场景展示。神经光谱场(NERF)使用神经网络将连续的 3D 场景结构嵌入网络重量内的连续 3D 场景结构,实现光化合成结果,然而,方法仅限于实际中效率低的单色优化设置。多平面平面图像(MPI)是代表场景的量方法,在不同的深度平面平面平面图中代表场面图,同时利用输入图像特征避免每摄氏面平面平面最优化。当前文学工作的主要前景是从单面面平面平面平面平面平面平面平面平面平面平面平面平面平面平面结构,同时将整面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面,我们研究,从一面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面,从一面,从一面平面平面平面平面平面平面平面平面平面平面平面平面图,从,从,从一面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面,将,将,从,从,将,将,将,将,将,将,将地平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面,</s>