Formulated as a conditional generation problem, face animation aims at synthesizing continuous face images from a single source image driven by a set of conditional face motion. Previous works mainly model the face motion as conditions with 1D or 2D representation (e.g., action units, emotion codes, landmark), which often leads to low-quality results in some complicated scenarios such as continuous generation and largepose transformation. To tackle this problem, the conditions are supposed to meet two requirements, i.e., motion information preserving and geometric continuity. To this end, we propose a novel representation based on a 3D geometric flow, termed facial flow, to represent the natural motion of the human face at any pose. Compared with other previous conditions, the proposed facial flow well controls the continuous changes to the face. After that, in order to utilize the facial flow for face editing, we build a synthesis framework generating continuous images with conditional facial flows. To fully take advantage of the motion information of facial flows, a hierarchical conditional framework is designed to combine the extracted multi-scale appearance features from images and motion features from flows in a hierarchical manner. The framework then decodes multiple fused features back to images progressively. Experimental results demonstrate the effectiveness of our method compared to other state-of-the-art methods.
翻译:作为有条件的生成问题,脸部动画旨在综合由一组有条件的面部运动驱动的单一来源图像的连续面部图像。先前的工作主要是将脸部运动作为1D或2D代表(如动作单位、情绪代码、里程碑)的条件模型,这往往导致某些复杂情景(如连续生成和放大变形)的低质量结果。为了解决这个问题,条件应该满足两个要求,即运动信息保护和几何连续性。为此,我们提议以3D几何流(称为面部流)为基础,以代表任何摆势的人类脸部自然运动为新的表述。与其他先前的条件相比,拟议的面部运动很好地控制面部变化。之后,为了利用面部流动进行面部编辑,我们建立了一个合成框架,产生连续的面部流图像。为了充分利用面部流动的动作信息,一个等级性有条件框架旨在将从从图像和运动流的多尺度外观特征(称为面部流)合并起来,以代表任何面部的自然运动。然后,框架与以往的其他条件相比,要很好地控制面部变化。之后,我们将多种组合的外观特征比对图像的实验方法进行分解。