Denoising diffusion probabilistic models (DDPMs) have been proven capable of synthesizing high-quality images with remarkable diversity when trained on large amounts of data. However, to our knowledge, few-shot image generation tasks have yet to be studied with DDPM-based approaches. Modern approaches are mainly built on Generative Adversarial Networks (GANs) and adapt models pre-trained on large source domains to target domains using a few available samples. In this paper, we make the first attempt to study when do DDPMs overfit and suffer severe diversity degradation as training data become scarce. Then we fine-tune DDPMs pre-trained on large source domains to solve the overfitting problem when training data is limited. Although the directly fine-tuned models accelerate convergence and improve generation quality and diversity compared with training from scratch, they still fail to retain some diverse features and can only produce coarse images. Therefore, we design a DDPM pairwise adaptation (DDPM-PA) approach to optimize few-shot DDPM domain adaptation. DDPM-PA efficiently preserves information learned from source domains by keeping the relative pairwise distances between generated samples during adaptation. Besides, DDPM-PA enhances the learning of high-frequency details from source models and limited training data. DDPM-PA further improves generation quality and diversity and achieves results better than current state-of-the-art GAN-based approaches. We demonstrate the effectiveness of our approach on a series of few-shot image generation tasks qualitatively and quantitatively.
翻译:事实证明,在对大量数据进行培训时,能够将高质量的图像与显著多样性结合起来,并具有显著的多样性;然而,据我们所知,尚未用基于DDPM的方法研究少量图像生成任务;现代方法主要以General Aversarial Networks(GANs)为基础,在大源域预先培训的模型中利用少数现有样本将大源域的模型调整为目标领域。在本文件中,我们第一次尝试在DDPMs过度适应时研究高品质图像,并在培训数据数量稀少时使多样性严重退化。然后,我们在大源域预先对DDPMS进行质量培训,以便在培训数据有限时解决问题。虽然直接调整的模型加快了趋同并提高了生成质量和多样性,但与从零开始的培训相比,它们仍然无法保留某些不同的特征,只能产生粗糙的图像。因此,我们设计了DDPM-PM的配对式适应方法,以优化几发DPM域适应。 DDPM-PA通过保持相对的距离,从源域中获取的DDP-PA数据,从而改进了目前数据源域中的数据质量模型的升级,从而改进了DDPD-PA-PA的升级改进了D-PA的升级和数据来源的升级结果。</s>