Robust face reconstruction from monocular image in general lighting conditions is challenging. Methods combining deep neural network encoders with differentiable rendering have opened up the path for very fast monocular reconstruction of geometry, lighting and reflectance. They can also be trained in self-supervised manner for increased robustness and better generalization. However, their differentiable rasterization based image formation models, as well as underlying scene parameterization, limit them to Lambertian face reflectance and to poor shape details. More recently, ray tracing was introduced for monocular face reconstruction within a classic optimization-based framework and enables state-of-the art results. However optimization-based approaches are inherently slow and lack robustness. In this paper, we build our work on the aforementioned approaches and propose a new method that greatly improves reconstruction quality and robustness in general scenes. We achieve this by combining a CNN encoder with a differentiable ray tracer, which enables us to base the reconstruction on much more advanced personalized diffuse and specular albedos, a more sophisticated illumination model and a plausible representation of self-shadows. This enables to take a big leap forward in reconstruction quality of shape, appearance and lighting even in scenes with difficult illumination. With consistent face attributes reconstruction, our method leads to practical applications such as relighting and self-shadows removal. Compared to state-of-the-art methods, our results show improved accuracy and validity of the approach.
翻译:在一般照明条件下,从单视图像中进行坚硬的面部重建十分艰巨。 将深神经网络编码器和不同图像结合起来的方法已经打开了非常快速单视重建几何、照明和反射的路径。 他们也可以接受自我监督的培训,以提高稳健性和更好的概括性。 但是,他们可以自我监督地进行自我监督的方式培训,以提高稳健性和更好的概括性。 然而,他们以不同的光谱为基础的图像形成模型,以及基本的场景参数化,限制他们为兰贝斯面部反射和形状细节差。 最近,在经典的以优化为基础的框架内,为单视面部面部重建引入了光线追踪方法。 然而,以优化为基础的方法本身本来就很慢,缺乏稳健。 在本文中,我们的工作建立在上述方法之上,提出了新的方法,大大提高了总体的重建质量和稳健健性。 我们通过将CNN的图像与不同的射线追踪器结合起来,从而使我们能够把重建建立在更先进的个化扩散和视觉的反射线上。 一个更复杂的污点模型,更精密的光化模型,甚至可以令人信服地反映自我阴影的自我清除的自我的图像。 这可以使我们的重建质量在现实的形状上呈现一个大的走向上,使得重建的升级的升级的自我走向。