Recent 3D face reconstruction methods reconstruct the entire head compared to earlier approaches which only model the face. Although these methods accurately reconstruct facial features, they do not explicitly regulate the upper part of the head. Extracting information about this part of the head is challenging due to varying degrees of occlusion by hair. We present a novel approach for modeling the upper head by removing occluding hair and reconstructing the skin, revealing information about the head shape. We introduce three objectives: 1) a dice consistency loss that enforces similarity between the overall head shape of the source and rendered image, 2) a scale consistency loss to ensure that head shape is accurately reproduced even if the upper part of the head is not visible, and 3) a 71 landmark detector trained using a moving average loss function to detect additional landmarks on the head. These objectives are used to train an encoder in an unsupervised manner to regress FLAME parameters from in-the-wild input images. Our unsupervised 3DMM model achieves state-of-the-art results on popular benchmarks and can be used to infer the head shape, facial features, and textures for direct use in animation or avatar creation.
翻译:最近 3D 面部重建方法重建了整个头部, 与之前只模拟脸部的方法相比。 虽然这些方法精确地重建面部特征, 但没有明确地调节头部的上部。 摘取关于头部这一部分的信息具有挑战性, 原因是头发隔开程度不同。 我们展示了一种新的方法, 通过摘除黑发和皮肤重建头部来模拟头部, 揭示头部形状的信息。 我们引入了三个目标 :(1) 骰子一致性损失, 使源头整体形状与成像相近; (2) 规模一致性损失, 以确保头部的上部形状能够准确复制, 即使头部上部不可见; 和 (3) 71个里程碑探测器, 使用移动平均损失功能来探测头部上的其他标志。 这些目标被用来以不受监督的方式训练一个编码器, 以便从输入图像中反向 FLAME 参数。 我们未被监督的 3DMM 模型在流行的基准上取得了最新的结果, 并且可以用来推断头部形状、 面部特征和纹素状, 用于直接制作动画。