Pose-conditioned convolutional generative models struggle with high-quality 3D-consistent image generation from single-view datasets, due to their lack of sufficient 3D priors. Recently, the integration of Neural Radiance Fields (NeRFs) and generative models, such as Generative Adversarial Networks (GANs), has transformed 3D-aware generation from single-view images. NeRF-GANs exploit the strong inductive bias of 3D neural representations and volumetric rendering at the cost of higher computational complexity. This study aims at revisiting pose-conditioned 2D GANs for efficient 3D-aware generation at inference time by distilling 3D knowledge from pretrained NeRF-GANS. We propose a simple and effective method, based on re-using the well-disentangled latent space of a pre-trained NeRF-GAN in a pose-conditioned convolutional network to directly generate 3D-consistent images corresponding to the underlying 3D representations. Experiments on several datasets demonstrate that the proposed method obtains results comparable with volumetric rendering in terms of quality and 3D consistency while benefiting from the superior computational advantage of convolutional networks. The code will be available at: https://github.com/mshahbazi72/NeRF-GAN-Distillation
翻译:对于从单视角数据集生成高质量三维连贯图像,姿态条件下的卷积生成模型存在困难,原因在于它们缺乏足够的三维先验知识。最近,Neural Radiance Fields(神经辐射场,简称NeRF)和生成式对抗网络(GAN)的集成,已经将从单视角图像感知三维生成进行了全面的转型。NeRF-GAN利用了三维神经表示和三维体素渲染的强归纳偏置,但是代价是更高的计算复杂度。本研究旨在通过从预训练的NeRF-GAN蒸馏三维知识,以实现卷积神经网络的姿态条件下高效的三维感知生成。我们提出了一种简单有效的方法,基于在姿态条件下的卷积网络中重复使用预先训练的NeRF-GAN的良好解缠结的潜空间,直接生成与底层三维表示相对应的三维连贯图像。在几个数据集上的实验表明,所提出的方法在质量和三维连贯性方面的表现可与体素渲染相媲美,同时也受益于卷积网络更优越的计算优势。代码可在https://github.com/mshahbazi72/NeRF-GAN-Distillation中获得。