Denoising diffusion probabilistic models (DDPMs) have been proven capable of synthesizing high-quality images with remarkable diversity when trained on large amounts of data. However, to our knowledge, few-shot image generation tasks have yet to be studied with DDPM-based approaches. Modern approaches are mainly built on Generative Adversarial Networks (GANs) and adapt models pre-trained on large source domains to target domains using a few available samples. In this paper, we make the first attempt to study when do DDPMs overfit and suffer severe diversity degradation as training data become scarce. Then we propose to adapt DDPMs pre-trained on large source domains to target domains using limited data. Our results show that utilizing knowledge from pre-trained DDPMs can significantly accelerate convergence and improve the quality and diversity of the generated images. Moreover, we propose a DDPM-based pairwise similarity loss to preserve the relative distances between generated samples during domain adaptation. In this way, we further improve the generation diversity of the proposed DDPM-based approaches. We demonstrate the effectiveness of our approaches qualitatively and quantitatively on a series of few-shot image generation tasks and achieve results better than current state-of-the-art GAN-based approaches in quality and diversity.
翻译:事实证明,在对大量数据进行培训时,可以将高质量的图像与显著多样性结合起来,并进行大量数据培训,但据我们了解,尚未用DDPM方法研究少量图像生成任务,现代方法主要以GANs为主,在大源域预先培训的模型使用少数现有样本,以目标领域为对象。在本文件中,我们第一次尝试在DDPMs过度适应和遭受严重多样性退化时进行研究,因为培训数据越来越少。然后,我们提议将大源域预先培训的DDPMs调整到使用有限数据的目标领域。我们的结果显示,利用预先培训的DDPMs的知识可以大大加快趋同,提高生成图像的质量和多样性。此外,我们提议以DDPM为主的相似性损失为主,以保持所生成的样本之间的相对距离。我们以此方式进一步改进拟议DDPM方法的生成多样性。我们展示了我们采用的方法在质量上和数量上都比GAN制图像生成结果更佳的一系列方法的有效性。