Recent advances in neural radiance fields have enabled the high-fidelity 3D reconstruction of complex scenes for novel view synthesis. However, it remains underexplored how the appearance of such representations can be efficiently edited while maintaining photorealism. In this work, we present PaletteNeRF, a novel method for photorealistic appearance editing of neural radiance fields (NeRF) based on 3D color decomposition. Our method decomposes the appearance of each 3D point into a linear combination of palette-based bases (i.e., 3D segmentations defined by a group of NeRF-type functions) that are shared across the scene. While our palette-based bases are view-independent, we also predict a view-dependent function to capture the color residual (e.g., specular shading). During training, we jointly optimize the basis functions and the color palettes, and we also introduce novel regularizers to encourage the spatial coherence of the decomposition. Our method allows users to efficiently edit the appearance of the 3D scene by modifying the color palettes. We also extend our framework with compressed semantic features for semantic-aware appearance editing. We demonstrate that our technique is superior to baseline methods both quantitatively and qualitatively for appearance editing of complex real-world scenes.
翻译:神经光亮场最近的进展使得3D重建复杂场景以进行新视觉合成。 但是,它仍然未能充分探索如何在保持光现实主义的同时高效地编辑这些表象的外观。 在此工作中,我们展示了PaletteNeRF, 这是基于 3D 色分解的神经光光场光化编辑的新颖方法。 我们的方法分解了3D 点的外观, 将每个3D点的外观变成一个线性组合, 即由一组 NERF 型功能定义的3D 分解, 并共享于全场。 虽然我们基于调色的基底是视不独立的, 我们还预测了一种以观为依存的功能来捕捉彩色残余( 例如, 视觉阴影 ) 。 在培训过程中, 我们共同优化基底功能和色调调, 我们还引入了新调调的调控调器, 以鼓励以空间分解的调调调。 我们的方法允许用户通过修改调色调调调色板来高效地编辑三D场景的外观景。 我们还用高压的镜像框架, 展示了我们精精精度的精度的图像。