DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis

Synthesizing high-quality realistic images from text descriptions is a challenging task. Almost all existing text-to-image Generative Adversarial Networks employ stacked architecture as the backbone. They utilize cross-modal attention mechanisms to fuse text and image features, and introduce extra networks to ensure text-image semantic consistency. In this work, we propose a much simpler, but more effective text-to-image model than previous works. Corresponding to the above three limitations, we propose: 1) a novel one-stage text-to-image backbone which is able to synthesize high-quality images directly by one pair of generator and discriminator, 2) a novel fusion module called deep text-image fusion block which deepens the text-image fusion process in generator, 3) a novel target-aware discriminator composed of matching-aware gradient penalty and one-way output which promotes the generator to synthesize more realistic and text-image semantic consistent images without introducing extra networks. Compared with existing text-to-image models, our proposed method (i.e., DF-GAN) is simpler but more efficient to synthesize realistic and text-matching images and achieves better performance. Extensive experiments on both Caltech-UCSD Birds 200 and COCO datasets demonstrate the superiority of the proposed model in comparison to state-of-the-art models.