The literature has proposed several methods to finetune pretrained GANs on new datasets, which typically results in higher performance compared to training from scratch, especially in the limited-data regime. However, despite the apparent empirical benefits of GAN pretraining, its inner mechanisms were not analyzed in-depth, and understanding of its role is not entirely clear. Moreover, the essential practical details, e.g., selecting a proper pretrained GAN checkpoint, currently do not have rigorous grounding and are typically determined by trial and error. This work aims to dissect the process of GAN finetuning. First, we show that initializing the GAN training process by a pretrained checkpoint primarily affects the model's coverage rather than the fidelity of individual samples. Second, we explicitly describe how pretrained generators and discriminators contribute to the finetuning process and explain the previous evidence on the importance of pretraining both of them. Finally, as an immediate practical benefit of our analysis, we describe a simple recipe to choose an appropriate GAN checkpoint that is the most suitable for finetuning to a particular target task. Importantly, for most of the target tasks, Imagenet-pretrained GAN, despite having poor visual quality, appears to be an excellent starting point for finetuning, resembling the typical pretraining scenario of discriminative computer vision models.
翻译:文献提出了在新的数据集上微调经过预先训练的GAN系统的方法,这些方法通常比从零开始的培训,特别是在有限的数据制度中,产生更高的业绩,然而,尽管GAN系统预培训显然具有经验上的好处,但其内部机制没有进行深入分析,对其作用的理解也不完全清楚,此外,基本的实际细节,例如选择经过适当事先训练的GAN检查站,目前没有严格的基础,通常由试验和错误决定。这项工作的目的是将GAN的微调过程分解开来。首先,我们表明,通过预先训练的检查站开始GAN培训过程,主要影响模型的覆盖范围,而不是个人样品的忠实性。第二,我们明确描述预先训练的发电机和歧视者如何有助于微调过程,并解释以前关于他们两人接受过预先训练的重要性的证据。最后,作为我们分析的直接实际好处,我们描述了选择一个最适于调整特定目标的适当的GAN检查站的简单方法。对于大多数目标任务来说,图像网前的典型的GAN模型看来是极差的,尽管其模范式的GAN模型看来是精准的。