Generative models have been successfully used for generating realistic signals. Because the likelihood function is typically intractable in most of these models, the common practice is to use "implicit" models that avoid likelihood calculation. However, it is hard to obtain theoretical guarantees for such models. In particular, it is not understood when they can globally optimize their non-convex objectives. Here we provide such an analysis for the case of Maximum Mean Discrepancy (MMD) learning of generative models. We prove several optimality results, including for a Gaussian distribution with low rank covariance (where likelihood is inapplicable) and a mixture of Gaussians. Our analysis shows that that the MMD optimization landscape is benign in these cases, and therefore gradient based methods will globally minimize the MMD objective.
翻译:由于可能性功能在大多数这些模型中通常难以找到,通常的做法是使用“隐含”模型,避免进行概率计算。然而,很难为这些模型获得理论保证。特别是,当这些模型能够在全球优化其非曲线目标时,人们并不理解这些模型的理论保证。我们在这里为了解基因模型的最大平均值差异提供了这样的分析。我们证明,在高斯分布上取得了一些最佳效果,包括低级共变(在可能性不适用的情况下)和高斯人混合。我们的分析表明,在这些情况下,MMD优化景观是良性的,因此,基于梯度的方法将在全球范围内最大限度地减少MMD目标。