Photo-realistic modeling and rendering of fuzzy objects with complex opacity are critical for numerous immersive VR/AR applications, but it suffers from strong view-dependent brightness, color. In this paper, we propose a novel scheme to generate opacity radiance fields with a convolutional neural renderer for fuzzy objects, which is the first to combine both explicit opacity supervision and convolutional mechanism into the neural radiance field framework so as to enable high-quality appearance and global consistent alpha mattes generation in arbitrary novel views. More specifically, we propose an efficient sampling strategy along with both the camera rays and image plane, which enables efficient radiance field sampling and learning in a patch-wise manner, as well as a novel volumetric feature integration scheme that generates per-patch hybrid feature embeddings to reconstruct the view-consistent fine-detailed appearance and opacity output. We further adopt a patch-wise adversarial training scheme to preserve both high-frequency appearance and opacity details in a self-supervised framework. We also introduce an effective multi-view image capture system to capture high-quality color and alpha maps for challenging fuzzy objects. Extensive experiments on existing and our new challenging fuzzy object dataset demonstrate that our method achieves photo-realistic, globally consistent, and fined detailed appearance and opacity free-viewpoint rendering for various fuzzy objects.
翻译:具有复杂不透明性的照片现实模型和模糊天体的翻版对于许多闪烁 VR/AR 应用程序至关重要,但它有很强的视觉光度和颜色。 在本文中,我们提出一个新的方案,以产生不透明光亮场,为模糊天体提供一个卷发神经造型器,这是第一个将显性不透明监督与共变机制结合到神经光亮场框架中,以便在任意的新观点中保持高质量外观和全球一致的阿尔法网友生成。更具体地说,我们提出一个高效的取样战略,同时配有摄影射线和图像平面,使高效的光亮场采样和以近似的方式学习,以及一个新型的体积特性集成器,产生超强的混合功能,以重建视觉一致的精细细的外观外观和不透明输出。我们还进一步采用了一种不完全对称的对抗性培训计划,以便在一个自我超强的框架内保存高频率外观的外观和不透明的细节。我们还引入了一种有效的多视角的多视角实地图像详细图像采集系统,以高品质、具有挑战性地展示我们目前高品质的视野的图像。