We consider restricted Boltzmann machine (RBMs) trained over an unstructured dataset made of blurred copies of definite but unavailable ``archetypes'' and we show that there exists a critical sample size beyond which the RBM can learn archetypes, namely the machine can successfully play as a generative model or as a classifier, according to the operational routine. In general, assessing a critical sample size (possibly in relation to the quality of the dataset) is still an open problem in machine learning. Here, restricting to the random theory, where shallow networks suffice and the grand-mother cell scenario is correct, we leverage the formal equivalence between RBMs and Hopfield networks, to obtain a phase diagram for both the neural architectures which highlights regions, in the space of the control parameters (i.e., number of archetypes, number of neurons, size and quality of the training set), where learning can be accomplished. Our investigations are led by analytical methods based on the statistical-mechanics of disordered systems and results are further corroborated by extensive Monte Carlo simulations.
翻译:我们认为,在非结构化的数据集中训练了有限的Boltzmann机器(RBMs),该机器由明确但不可使用的“无政府类型”的模糊副本组成,因此,我们表明,存在一个关键的样本规模,使成果管理制能够学习古型,即机器能够按照操作常规成功地作为基因模型或分类机发挥作用。一般而言,评估关键的样本规模(可能与数据集的质量有关)仍然是机器学习中的一个未解决的问题。这里,在随机理论中,浅线网络已经足够,母体细胞假设是正确的,我们利用按成果管理网络和Hopfield网络之间的正式等值,以获得神经结构的阶段图,在控制参数的空间(即,成型类型数量、神经元数量、培训组合的规模和质量)中,在能够完成学习的地方,突出区域(即,成型类型数量、神经元数量、规模和质量)的神经结构。我们的调查以基于统计-机械系统的分析方法进行,结果得到广泛的蒙特卡洛模拟的进一步证实。