3D-aware image generative modeling aims to generate 3D-consistent images with explicitly controllable camera poses. Recent works have shown promising results by training neural radiance field (NeRF) generators on unstructured 2D images, but still can not generate highly-realistic images with fine details. A critical reason is that the high memory and computation cost of volumetric representation learning greatly restricts the number of point samples for radiance integration during training. Deficient sampling not only limits the expressive power of the generator to handle fine details but also impedes effective GAN training due to the noise caused by unstable Monte Carlo sampling. We propose a novel approach that regulates point sampling and radiance field learning on 2D manifolds, embodied as a set of learned implicit surfaces in the 3D volume. For each viewing ray, we calculate ray-surface intersections and accumulate their radiance generated by the network. By training and rendering such radiance manifolds, our generator can produce high quality images with realistic fine details and strong visual 3D consistency.
翻译:3D-觉察到的图像基因模型旨在生成3D相容的图像,并配有明确可控的相机。最近的工作通过在非结构化的 2D 图像上培训神经光场(NERF) 生成器,显示了有希望的成果,但仍不能产生高度现实的图像,细细的详情。一个关键的原因是,体积显示学习的高记忆和计算成本极大地限制了在培训期间进行光度整合的点样本数量。缺乏取样不仅限制了发电机处理精细细节的表达力,而且由于蒙特卡洛取样不稳定造成的噪音,妨碍了有效的GAN培训。我们提出了一个新颖的方法,规范了在2D 柱上的点采样和光场学习,作为一套在3D 卷中体现的学习的隐含表层。对于每个观看的射线,我们计算光-表面的交叉点,并积累网络产生的亮度。通过培训和制作这些光度的元件,我们的发电机可以产生高品质的图像,并具有现实的精细和强烈的视觉3D 一致性。