Generative adversarial networks (GANs) are among the most successful models for learning high-complexity, real-world distributions. However, in theory, due to the highly non-convex, non-concave landscape of the minmax training objective, GAN remains one of the least understood deep learning models. In this work, we formally study how GANs can efficiently learn certain hierarchically generated distributions that are close to the distribution of real-life images. We prove that when a distribution has a structure that we refer to as Forward Super-Resolution, then simply training generative adversarial networks using stochastic gradient descent ascent (SGDA) can learn this distribution efficiently, both in sample and time complexities. We also provide empirical evidence that our assumption "forward super-resolution" is very natural in practice, and the underlying learning mechanisms that we study in this paper (to allow us efficiently train GAN via SGDA in theory) simulates the actual learning process of GANs on real-world problems.
翻译:生成对抗网络(GANs)是学习高复杂度、现实世界分布中最成功的模型之一。然而,理论上,由于最小最大训练目标的高度非凸、非凹性质,GAN仍然是最不理解的深度学习模型之一。在这项工作中,我们正式研究了GAN如何有效地学习某些层次生成的分布,这些分布接近于真实世界图像的分布。我们证明,当一个分布具有我们称之为“前向超分辨率”结构时,简单地使用随机梯度下降上升(SGDA)训练生成对抗网络可以有效地学习该分布,无论是在样本复杂度还是在时间复杂度上。我们还提供了经验证据,证明了我们的假设“前向超分辨率”在实践中非常自然,并且我们在本文中研究的底层学习机制(通过SGDA理论上允许我们有效地训练GAN)模拟了GAN在现实问题上的实际学习过程。