One-shot generative domain adaption aims to transfer a pre-trained generator on one domain to a new domain using one reference image only. However, it remains very challenging for the adapted generator (i) to generate diverse images inherited from the pre-trained generator while (ii) faithfully acquiring the domain-specific attributes and styles of the reference image. In this paper, we present a novel one-shot generative domain adaption method, i.e., DiFa, for diverse generation and faithful adaptation. For global-level adaptation, we leverage the difference between the CLIP embedding of reference image and the mean embedding of source images to constrain the target generator. For local-level adaptation, we introduce an attentive style loss which aligns each intermediate token of adapted image with its corresponding token of the reference image. To facilitate diverse generation, selective cross-domain consistency is introduced to select and retain the domain-sharing attributes in the editing latent $\mathcal{W}+$ space to inherit the diversity of pre-trained generator. Extensive experiments show that our method outperforms the state-of-the-arts both quantitatively and qualitatively, especially for the cases of large domain gaps. Moreover, our DiFa can easily be extended to zero-shot generative domain adaption with appealing results. Code is available at https://github.com/1170300521/DiFa.
翻译:一次性基因化域变适应旨在将一个域的预培训生成器转移到仅使用一个参考图像的新域,然而,对经调整的生成器来说,它仍然非常具有挑战性:(一) 生成从经过训练的生成器继承的不同图像,同时(二) 忠实地获取参考图像的域特性和风格。在本文中,我们提出了一个新型的一次性基因变适应方法,即Difa,用于不同的生成和忠实的适应。关于全球层面的适应,我们利用CLIP嵌入参考图像与平均嵌入源图像以限制目标生成器之间的差别。对于地方一级的适应,我们引入了一种细微的风格损失,将每个经过调整的图像中间符号与其相应的参考图像符号相匹配。为了便利多样化的生成,我们引入了选择性的跨部一致性,以选择和保留编辑潜值 $mathcal{W ⁇ $$$$$$$$$$$$$$$$$$$$$$ 来继承事先训练的发电机的多样性。广泛的实验表明,我们的方法在定量和定性上都优于状态,特别是用于大规模域域的变换。