Despite the recent developments in 3D Face Reconstruction from occluded and noisy face images, the performance is still unsatisfactory. Moreover, most existing methods rely on additional dependencies, posing numerous constraints over the training procedure. Therefore, we propose a Self-Supervised RObustifying GUidancE (ROGUE) framework to obtain robustness against occlusions and noise in the face images. The proposed network contains 1) the Guidance Pipeline to obtain the 3D face coefficients for the clean faces and 2) the Robustification Pipeline to acquire the consistency between the estimated coefficients for occluded or noisy images and the clean counterpart. The proposed image- and feature-level loss functions aid the ROGUE learning process without posing additional dependencies. To facilitate model evaluation, we propose two challenging occlusion face datasets, ReaChOcc and SynChOcc, containing real-world and synthetic occlusion-based face images for robustness evaluation. Also, a noisy variant of the test dataset of CelebA is produced for evaluation. Our method outperforms the current state-of-the-art method by large margins (e.g., for the perceptual errors, a reduction of 23.8% for real-world occlusions, 26.4% for synthetic occlusions, and 22.7% for noisy images), demonstrating the effectiveness of the proposed approach. The occlusion datasets and the corresponding evaluation code are released publicly at https://github.com/ArcTrinity9/Datasets-ReaChOcc-and-SynChOcc.
翻译:尽管3D Face Reformation最近从隐蔽和吵闹的面部图像中取得了发展,但业绩仍然不尽如人意。此外,大多数现有方法依赖更多的依赖性,对培训程序造成许多限制。因此,我们提议了一个自我强化的对 GUIdancE (ROGUE) 框架,以获得对面图像中隐蔽和噪音的稳健性。拟议网络包含1) 指导管道,以获得清洁面部的3D面部系数;2) 强化管道,以获得隐蔽或噪音图像估计系数与清洁对应方之间的一致性。拟议的图像和地平级损失函数有助于ROGUE的学习进程,而不会造成额外的依赖性。为了便于模型评估,我们提议了两种具有挑战性的隐蔽面数据集,ReaCHOcc和SynCHOCC, 包含真实世界和合成的隐蔽面图像,此外,CelibeA测试数据集的扰动变式为评估。我们的方法超越了目前水平-%的精确度。