The research topic of sketch-to-portrait generation has witnessed a boost of progress with deep learning techniques. The recently proposed StyleGAN architectures achieve state-of-the-art generation ability but the original StyleGAN is not friendly for sketch-based creation due to its unconditional generation nature. To address this issue, we propose a direct conditioning strategy to better preserve the spatial information under the StyleGAN framework. Specifically, we introduce Spatially Conditioned StyleGAN (SC-StyleGAN for short), which explicitly injects spatial constraints to the original StyleGAN generation process. We explore two input modalities, sketches and semantic maps, which together allow users to express desired generation results more precisely and easily. Based on SC-StyleGAN, we present DrawingInStyles, a novel drawing interface for non-professional users to easily produce high-quality, photo-realistic face images with precise control, either from scratch or editing existing ones. Qualitative and quantitative evaluations show the superior generation ability of our method to existing and alternative solutions. The usability and expressiveness of our system are confirmed by a user study.
翻译:通过深层学习技术,素描到光谱生成的研究主题取得了进步。最近提出的StyleGAN结构实现了最先进的生成能力,但原StyleGAN因其无条件的生成性质,对素描基础的生成并不友好。为解决这一问题,我们提出了一个直接调节战略,以更好地保存StyleGAN框架下的空间信息。具体地说,我们引入了空间调控StyleGAN(Sc-StyleGAN)(StyleGAN)短片,这明确为原StyleGAN的生成过程注入了空间限制。我们探索了两种输入模式,即素描和语义图,使用户能够更精确和容易地表达所期望的生成结果。基于SC-StyleGAN,我们介绍“绘图Intyles”,这是为非专业用户制作高质量、具有照片现实面像的新型界面,以便从刮痕或编辑现有图像得到精确控制。定性和定量评价表明我们的方法对于现有和替代解决办法具有优越的一代能力。用户研究证实了我们系统的实用性和明确性。