Despite the recent developments in 3D Face Reconstruction from occluded and noisy face images, the performance is still unsatisfactory. One of the main challenges is to handle moderate to heavy occlusions in the face images. In addition, the noise in the face images inhibits the correct capture of facial attributes, thus needing to be reliably addressed. Moreover, most existing methods rely on additional dependencies, posing numerous constraints over the training procedure. Therefore, we propose a Self-Supervised RObustifying GUidancE (ROGUE) framework to obtain robustness against occlusions and noise in the face images. The proposed network contains 1) the Guidance Pipeline to obtain the 3D face coefficients for the clean faces, and 2) the Robustification Pipeline to acquire the consistency between the estimated coefficients for occluded or noisy images and the clean counterpart. The proposed image- and feature-level loss functions aid the ROGUE learning process without posing additional dependencies. On the three variations of the test dataset of CelebA: rational occlusions, delusional occlusions, and noisy face images, our method outperforms the current state-of-the-art method by large margins (e.g., for the shape-based 3D vertex errors, a reduction from 0.146 to 0.048 for rational occlusions, from 0.292 to 0.061 for delusional occlusions and from 0.269 to 0.053 for the noise in the face images), demonstrating the effectiveness of the proposed approach.
翻译:尽管在3D面部重建从隐蔽和吵闹的面部图像中最近出现了发展动态,但业绩仍然不能令人满意。主要挑战之一是处理面部图像中中度至重度的隐蔽性。此外,面部图像中的噪音抑制了正确捕捉面部特征的正确性,因此需要可靠地加以解决。此外,大多数现有方法依赖更多的依赖性,对培训程序造成许多制约。因此,我们提议建立一个自我超常的ROBIVI化 GUIDA(ROGUE)框架,以获得抵御面部图像中的隐蔽性和噪音的稳健性。拟议网络包含:1) 用于获取3D脸部清晰面部的隐隐蔽性系数的指导管道;2) 固化管道,以取得隐蔽或噪音图像与清洁对应方之间估计系数的一致性。 拟议的图像和地平级损失功能有助于ROGUE学习进程,而不会产生额外的依赖性。 CelibA测试数据集的三个变式:合理隐蔽性、错觉隐蔽性DLI3-D图像,以及从当前平面平面平面平面图像从当前平面图到0.0.0的方法。