In this paper, we study the task of hallucinating an authentic high-resolution (HR) face from an occluded thumbnail. We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN, which exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces (16*16 pixels). Pro-UIGAN iteratively (1) estimates facial geometry priors for low-resolution (LR) faces and (2) acquires non-occluded HR face images under the guidance of the estimated priors. Our multi-stage hallucination network super-resolves and inpaints occluded LR faces in a coarse-to-fine manner, thus reducing unwanted blurriness and artifacts significantly. Specifically, we design a novel cross-modal transformer module for facial priors estimation, in which an input face and its landmark features are formulated as queries and keys, respectively. Such a design encourages joint feature learning across the input facial and landmark features, and deep feature correspondences will be discovered by attention. Thus, facial appearance features and facial geometry priors are learned in a mutual promotion manner. Extensive experiments demonstrate that our Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks, i.e., face alignment, face parsing, face recognition and expression classification, compared with other state-of-the-art (SotA) methods.
翻译:在本文中,我们研究了从一个隐蔽的缩略图中将真实的高分辨率面部(HR)做成幻觉的任务。我们建议建立一个名为Pro-UIGAN的多阶段渐进式抽取和涂抹生成生成反反对映网络,称为Pro-UIGAN,利用面部几何学前科补充和加增(8*)隐蔽和微小面部(16*16像素)。Pro-UIGAN迭代(1)估计低分辨率面部面部的面部几何学前科,(2)在估计前身的指导下获取非隐蔽的HR脸部图像。我们的多阶段幻觉网络超级溶胶和印印出生成生成生成反射反射反射网络,从而显著减少不必要的模糊和人工合成(8*)面部面部面部(16*16像素)。我们设计了一个新的跨模式变形模型,其中输入面部面部面部和标志性特征分别作为查询和钥匙。这种设计鼓励在投入面部和标志性面部对面面的图像进行联合学习。我们所了解的面部面面部和面部面面面部的面部图像分析,从而显示了我们所了解的面面部的面部和面部的面部。