We introduce a new family of energy-based probabilistic graphical models for efficient unsupervised learning. Its definition is motivated by the control of the spin-glass properties of the Ising model described by the weights of Boltzmann machines. We use it to learn the Bars and Stripes dataset of various sizes and the MNIST dataset, and show how they quickly achieve the performance offered by standard methods for unsupervised learning. Our results indicate that the standard initialization of Boltzmann machines with random weights equivalent to spin-glass models is an unnecessary bottleneck in the process of training. Furthermore, this new family allows for very easy access to low-energy configurations, which points to new, efficient training algorithms. The simplest variant of such algorithms approximates the negative phase of the log-likelihood gradient with no Markov chain Monte Carlo sampling costs at all, and with an accuracy sufficient to achieve good learning and generalization.
翻译:我们引入了一套新的基于能源的概率性图象模型,以便高效、不受监督的学习。其定义的动机是控制由Boltzmann机器重量描述的Ising模型的脊椎玻璃特性。我们用它学习不同尺寸的条纹和条纹数据集和MNIST数据集,并展示它们如何迅速达到无监督学习的标准方法所提供的性能。我们的结果表明,在培训过程中,使用随机重量相当于脊椎模型的Boltzmann机器的标准初始化是一个不必要的瓶颈。此外,这个新系统允许非常容易地获得低能量配置,这指出了新的、有效的培训算法。这种算法的最简单变量接近了逻辑相似的梯度的负阶段,而没有Markov连锁的Monte Carlo取样成本,而且准确性足以实现良好的学习和概括化。