Inverse rendering methods aim to estimate geometry, materials and illumination from multi-view RGB images. In order to achieve better decomposition, recent approaches attempt to model indirect illuminations reflected from different materials via Spherical Gaussians (SG), which, however, tends to blur the high-frequency reflection details. In this paper, we propose an end-to-end inverse rendering pipeline that decomposes materials and illumination from multi-view images, while considering near-field indirect illumination. In a nutshell, we introduce the Monte Carlo sampling based path tracing and cache the indirect illumination as neural radiance, enabling a physics-faithful and easy-to-optimize inverse rendering method. To enhance efficiency and practicality, we leverage SG to represent the smooth environment illuminations and apply importance sampling techniques. To supervise indirect illuminations from unobserved directions, we develop a novel radiance consistency constraint between implicit neural radiance and path tracing results of unobserved rays along with the joint optimization of materials and illuminations, thus significantly improving the decomposition performance. Extensive experiments demonstrate that our method outperforms the state-of-the-art on multiple synthetic and real datasets, especially in terms of inter-reflection decomposition.
翻译:逆向渲染方法旨在从多角度RGB图像中估算几何、材质和照明。为了实现更好的分解效果,最近的方法尝试通过球型高斯函数(SG)模拟不同材质反射的间接照明,然而,这往往会模糊高频反射细节。本文提出了一种端到端逆向渲染流程,可以从多视角图像中分解材质和照明,同时考虑近场间接照明。简言之,我们引入基于蒙特卡罗采样的路径追踪,并将间接照明缓存为神经辐射,从而实现物理上可信且易于优化的逆向渲染方法。为了提高效率和实用性,我们利用SG表示平滑的环境照明,并应用重要性采样技术。为了监督未观察到方向的间接照明,我们在隐式的神经辐射和未观察到光线的路径追踪结果之间开发了一种新的辐射一致性约束,并通过材料和照明的联合优化,显著提高了分解性能。大量实验表明,我们的方法在多个合成和真实数据集上优于现有技术,特别是在反射分解方面。