Occlusions are a common occurrence in unconstrained face images. Single image 3D reconstruction from such face images often suffers from corruption due to the presence of occlusions. Furthermore, while a plurality of 3D reconstructions is plausible in the occluded regions, existing approaches are limited to generating only a single solution. To address both of these challenges, we present Diverse3DFace, which is specifically designed to simultaneously generate a diverse and realistic set of 3D reconstructions from a single occluded face image. It consists of three components: a global+local shape fitting process, a graph neural network-based mesh VAE, and a Determinantal Point Process based diversity promoting iterative optimization procedure. Quantitative and qualitative comparisons of 3D reconstruction on occluded faces show that Diverse3DFace can estimate 3D shapes that are consistent with the visible regions in the target image while exhibiting high, yet realistic, levels of diversity on the occluded regions. On face images occluded by masks, glasses, and other random objects, Diverse3DFace generates a distribution of 3D shapes having ~50% higher diversity on the occluded regions compared to the baselines. Moreover, our closest sample to the ground truth has ~40% lower MSE than the singular reconstructions by existing approaches.
翻译:在未受限制的面部图像中,隔离是常见的。从这种面部图像中重建的单一图像 3D 通常会因存在隔离而出现腐败。此外,虽然在隐蔽区域,3D重建的多元性是可行的,但现有办法仅限于产生单一的解决办法。为了应对这两个挑战,我们介绍了多样化3DFace, 专门设计它是为了同时产生一套多样化和现实的3D重建,由单一隐蔽的面部图像组成。它由三个组成部分组成:一个全球+地方形状安装过程,一个基于图形网络的神经元件VAE,一个基于多样性的震动点进程,促进迭代优化程序。对3D重建隐蔽面部的定量和定性比较表明,3DFDFace可以估计与目标图像中可见的区域相一致的3D形状,同时显示隐蔽区域高度、但现实的多样化程度。在面部、眼镜和其他随机物体所隐蔽的面部图像中,基于图形网络网状网状网状网状网状网状网状网状网状的M-40 级图状图状图状图状图状的分布比目前为最接近的40。