This paper presents a portrait stylization method designed for real-time mobile applications with limited style examples available. Previous learning based stylization methods suffer from the geometric and semantic gaps between portrait domain and style domain, which obstacles the style information to be correctly transferred to the portrait images, leading to poor stylization quality. Based on the geometric prior of human facial attributions, we propose to utilize geometric alignment to tackle this issue. Firstly, we apply Thin-Plate-Spline (TPS) on feature maps in the generator network and also directly to style images in pixel space, generating aligned portrait-style image pairs with identical landmarks, which closes the geometric gaps between two domains. Secondly, adversarial learning maps the textures and colors of portrait images to the style domain. Finally, geometric aware cycle consistency preserves the content and identity information unchanged, and deformation invariant constraint suppresses artifacts and distortions. Qualitative and quantitative comparison validate our method outperforms existing methods, and experiments proof our method could be trained with limited style examples (100 or less) in real-time (more than 40 FPS) on mobile devices. Ablation study demonstrates the effectiveness of each component in the framework.
翻译:本文展示了为实时移动应用设计、具有有限风格实例的肖像性化方法。 以往的基于学习的Styl化方法在肖像域和样式域之间存在几何和语义差异,这些差异妨碍了向肖像图像正确传输的风格信息,导致石质化质量差。 基于人类面貌属性的几何前期,我们建议利用几何校正来解决这一问题。 首先,我们在发电机网络的地貌地图上应用Thin-Plate-Spline(TPS),并直接用于像素空间图像的风格化,生成具有相同地标的对像式图像配对,从而缩小了两个域之间的几何差距。 其次,对立式学习将肖像的纹理和颜色绘制到样式域。 最后,以几何感知周期一致性保持内容和身份信息不变,以及变异性约束抑制了文物和扭曲。 定性和定量比较验证了我们的方法优于现有方法,并实验证明我们的方法可以用有限的风格示例(100个或更少个)在实时框架的移动设备中用有限的样式示例示例(超过40个)来演示。