Generating images from a single sample, as a newly developing branch of image synthesis, has attracted extensive attention. In this paper, we formulate this problem as sampling from the conditional distribution of a single image, and propose a hierarchical framework that simplifies the learning of the intricate conditional distributions through the successive learning of the distributions about structure, semantics and texture, making the process of learning and generation comprehensible. On this basis, we design ExSinGAN composed of three cascaded GANs for learning an explainable generative model from a given image, where the cascaded GANs model the distributions about structure, semantics and texture successively. ExSinGAN is learned not only from the internal patches of the given image as the previous works did, but also from the external prior obtained by the GAN inversion technique. Benefiting from the appropriate combination of internal and external information, ExSinGAN has a more powerful capability of generation and competitive generalization ability for the image manipulation tasks compared with prior works.
翻译:从单一样本中生成图像,作为新开发的图像合成分支,引起了广泛的注意。在本文中,我们将这一问题作为单一图像有条件分布的抽样来看待,并提出一个等级框架,通过连续学习结构、语义和纹理的分布,简化对复杂有条件分布的学习,使学习和生成过程可以理解。在此基础上,我们设计由三套级联GAN组成的ExSinGAN,从一个特定图像中学习一个可解释的基因化模型,在这个图像中,级联的GANs对结构、语义和纹理的分布相继进行模型。ExSinGAN不仅像以前的工作一样从特定图像的内部部分学习,而且从GAN的转换技术先前获得的外部学习。ExSinGAN从内部和外部信息的适当组合中受益,在与先前的工程相比,为图像处理任务创造和竞争通用能力方面拥有更大的能力。