Generative Adversarial Networks (GANs) have been widely applied in different scenarios thanks to the development of deep neural networks. The original GAN was proposed based on the non-parametric assumption of the infinite capacity of networks. However, it is still unknown whether GANs can fit the target distribution without any prior information. Due to the overconfident assumption, many issues remain unaddressed in GANs' training, such as non-convergence, mode collapses, gradient vanishing. Regularization and normalization are common methods of introducing prior information to stabilize training and improve discrimination. Although a handful number of regularization and normalization methods have been proposed for GANs, to the best of our knowledge, there exists no comprehensive survey that primarily focuses on objectives and development of these methods, apart from some in-comprehensive and limited scope studies. In this work, we conduct a comprehensive survey on the regularization and normalization techniques from different perspectives of GANs training. First, we systematically describe different perspectives of GANs training and thus obtain the different objectives of regularization and normalization. Based on these objectives, we propose a new taxonomy. Furthermore, we compare the performance of the mainstream methods on different datasets and investigate the applications of regularization and normalization techniques that have been frequently employed in state-of-the-art GANs. Finally, we highlight potential future directions of research in this domain. Code and studies related to the regularization and normalization of GANs in this work is summarized on https://github.com/iceli1007/GANs-Regularization-Review.
翻译:由于发展了深层神经网络,不同情景中广泛应用了生成的Adversarial网络(GANs),最初的GAN是根据对网络无限能力的非参数假设提出的,然而,除了一些不全面且范围有限的研究外,尚不清楚GANs是否能够在没有任何事先信息的情况下达到目标分布;由于过于自信的假设,GANs培训中许多问题仍然没有得到解决,如不兼容、模式崩溃、梯度消失等;正规化和正常化是事先介绍信息以稳定培训和改善歧视的常见方法;尽管为GANs提出了少量的正规化和正常化方法,以最佳的知识为基础,但目前还没有一项主要侧重于这些方法的目标和开发的全面调查;此外,我们在这项工作中,对从GANs培训的不同角度出发的正规化和正常化技术进行了全面调查;首先,我们系统地介绍了GANs培训的不同观点,从而获得了正规化和正常化的不同目标;根据这些目标,我们提出了对GANsrationality-Ransality应用的新方法和GANs。