We propose a Few-shot Dynamic Neural Radiance Field (FDNeRF), the first NeRF-based method capable of reconstruction and expression editing of 3D faces based on a small number of dynamic images. Unlike existing dynamic NeRFs that require dense images as input and can only be modeled for a single identity, our method enables face reconstruction across different persons with few-shot inputs. Compared to state-of-the-art few-shot NeRFs designed for modeling static scenes, the proposed FDNeRF accepts view-inconsistent dynamic inputs and supports arbitrary facial expression editing, i.e., producing faces with novel expressions beyond the input ones. To handle the inconsistencies between dynamic inputs, we introduce a well-designed conditional feature warping (CFW) module to perform expression conditioned warping in 2D feature space, which is also identity adaptive and 3D constrained. As a result, features of different expressions are transformed into the target ones. We then construct a radiance field based on these view-consistent features and use volumetric rendering to synthesize novel views of the modeled faces. Extensive experiments with quantitative and qualitative evaluation demonstrate that our method outperforms existing dynamic and few-shot NeRFs on both 3D face reconstruction and expression editing tasks. Our code and model will be available upon acceptance.
翻译:我们建议采用一个微小的动态神经辐射场(FDNERF),这是第一个基于NeRF的、能够根据少量动态图像对 3D 面孔进行重建和表达编辑的、以动态图像为基础进行重建和表达的方法。与现有的动态 NERF不同,这些动态面孔需要密集图像作为输入,并且只能为单一身份建模,我们的方法可以让不同的人用少量投入进行面部重建。与为静态场景建模而设计的最新微小镜头相比,拟议的FDNERF接受视觉不兼容的动态输入,支持任意面部表达编辑,即用输入之外的新的表达方式制作面部面部。为了处理动态输入之间的不一致,我们采用了一个设计完善的有条件特征扭曲模块(CFW),在2D 地貌空间进行有条件的扭曲,这也是身份适应和3D 限制。结果,不同表达方式的特征被转换成目标场景。然后,我们根据这些视觉一致的特征构建一个彩色场景区域,用数量缩图来综合模型面面面部的新观点。为了广泛进行实验,用定量和定性的实验,用定量和定性的模型在现有的格式编辑中显示时,将显示我们的方法格式将显示。