One of the training strategies of generative models is to minimize the Jensen--Shannon divergence between the model distribution and the data distribution. Since data distribution is unknown, generative adversarial networks (GANs) formulate this problem as a game between two models, a generator and a discriminator. The training can be formulated in the context of game theory and the local Nash equilibrium (LNE). It does not seem feasible to derive guarantees of stability or optimality for the existing methods. This optimization problem is far more challenging than the single objective setting. Here, we use the conjugate gradient method to reliably and efficiently solve the LNE problem in GANs. We give a proof and convergence analysis under mild assumptions showing that the proposed method converges to a LNE with three different learning rate update rules, including a constant learning rate. Finally, we demonstrate that the proposed method outperforms stochastic gradient descent (SGD) and momentum SGD in terms of best Frechet inception distance (FID) score and outperforms Adam on average. The code is available at \url{https://github.com/Hiroki11x/ConjugateGradient_GAN}.
翻译:基因模型的培训战略之一是尽量减少模型分布和数据分布之间的Jensen-Shannon差异; 由于数据分布未知, 基因对抗网络(GANs)将这一问题作为两种模型、 生成者和歧视者之间的游戏。 培训可以在游戏理论和当地纳什均衡(LNE)的背景下制定; 现有方法的稳定性或最佳性保障似乎不可行。 这个优化问题比单一目标设定要困难得多。 这里, 我们使用同源梯法可靠和高效地解决GANs的LNE问题。 我们在温和假设下提供了证据和趋同分析,表明拟议方法与LNE相融合,有三个不同的学习率更新规则,包括一个不变的学习率。 最后, 我们证明拟议方法在最佳Frechet起始距离(FID)分数和顶值亚当方面超越了SGDGT。 该代码平均可在以下查阅:\url{https://githu.com/Hiroki11_Congate_GARgate.