Energy-based modeling is a promising approach to unsupervised learning, which yields many downstream applications from a single model. The main difficulty in learning energy-based models with the "contrastive approaches" is the generation of samples from the current energy function at each iteration. Many advances have been made to accomplish this subroutine cheaply. Nevertheless, all such sampling paradigms run MCMC targeting the current model, which requires infinitely long chains to generate samples from the true energy distribution and is problematic in practice. This paper proposes an alternative approach to getting these samples and avoiding crude MCMC sampling from the current model. We accomplish this by viewing the evolution of the modeling distribution as (i) the evolution of the energy function, and (ii) the evolution of the samples from this distribution along some vector field. We subsequently derive this time-dependent vector field such that the particles following this field are approximately distributed as the current density model. Thereby we match the evolution of the particles with the evolution of the energy function prescribed by the learning procedure. Importantly, unlike Monte Carlo sampling, our method targets to match the current distribution in a finite time. Finally, we demonstrate its effectiveness empirically compared to MCMC-based learning methods.
翻译:以能源为基础的建模是一种有希望的不受监督的学习方法,它从单一模型中产生许多下游应用。学习以能源为基础的模型和“交替方法”的主要困难在于在每个迭代中从目前的能源功能中产生样品。为了实现这一次常规,已经取得了许多进展。然而,所有这些采样模式都以目前的模型为对象,而目前的模型则要求有无限长的链条从真正的能源分布中产生样品,在实践中有问题。本文件提出了获取这些样品和避免从当前模型中采集粗体质的MCMC取样的替代方法。我们通过将模型分布的演变看成(一) 能源功能的演变,以及(二) 从某些矢量场上这种分布的样品的演变来完成这项工作。我们随后从这个依赖时间的矢量域中得出这样的矢量场,即这个字段的粒子大致分布为目前的密度模型。我们通过将粒子的演进与学习程序所规定的能源功能的演变相匹配。重要的是,与蒙特卡洛取样不同,我们的方法目标在有限的时间内与目前的分布相匹配。我们展示了它的有效性经验学习的方法方法。