The wide applications of Generative adversarial networks benefit from the successful training methods, guaranteeing that an object function converges to the local minima. Nevertheless, designing an efficient and competitive training method is still a challenging task due to the cyclic behaviors of some gradient-based ways and the expensive computational cost of these methods based on the Hessian matrix. This paper proposed the adaptive Composite Gradients (ACG) method, linearly convergent in bilinear games under suitable settings. Theory and toy-function experiments suggest that our approach can alleviate the cyclic behaviors and converge faster than recently proposed algorithms. Significantly, the ACG method is not only used to find stable fixed points in bilinear games as well as in general games. The ACG method is a novel semi-gradient-free algorithm since it does not need to calculate the gradient of each step, reducing the computational cost of gradient and Hessian by utilizing the predictive information in future iterations. We conducted two mixture of Gaussians experiments by integrating ACG to existing algorithms with Linear GANs. Results show ACG is competitive with the previous algorithms. Realistic experiments on four prevalent data sets (MNIST, Fashion-MNIST, CIFAR-10, and CelebA) with DCGANs show that our ACG method outperforms several baselines, which illustrates the superiority and efficacy of our method.
翻译:创世对抗网络的广泛应用得益于成功的培训方法,保证对象功能与当地小型算法相融合。然而,设计高效和竞争性培训方法仍是一项艰巨的任务,因为某些梯度基方法的周期性行为以及基于赫西安矩阵的这些方法的昂贵计算成本,因此,设计高效和竞争性培训方法仍是一项具有挑战性的任务。本文建议采用适应性综合梯度梯度方法,在适当环境下双线游戏中线性趋同。理论和玩具功能实验表明,我们的方法可以减轻周期性行为,并比最近提议的算法更快。重要的是,ACG方法不仅用于在双线游戏和普通游戏中找到稳定的固定点。ACG方法是一种全新的半梯度计算法,因为它不需要计算每一步的梯度,通过利用预测性信息减少梯度和赫西的计算成本。我们进行了两种高比实验,将ACG与现有算法结合到GANs的现有算法。重要的是,ACG方法不仅用于在双线游戏和普通游戏中找到稳定的固定点。ACAGA-CA,结果展示了我们以往的ARCA-C-SAM 的基底底基数据。