The gap between the huge volumes of data needed to train artificial neural networks and the relatively small amount of data needed by their biological counterparts is a central puzzle in machine learning. Here, inspired by biological information-processing, we introduce a generalized Hopfield network where pairwise couplings between neurons are built according to Hebb's prescription for on-line learning and allow also for (suitably stylized) off-line sleeping mechanisms. Moreover, in order to retain a learning framework, here the patterns are not assumed to be available, instead, we let the network experience solely a dataset made of a sample of noisy examples for each pattern. We analyze the model by statistical-mechanics tools and we obtain a quantitative picture of its capabilities as functions of its control parameters: the resulting network is an associative memory for pattern recognition that learns from examples on-line, generalizes and optimizes its storage capacity by off-line sleeping. Remarkably, the sleeping mechanisms always significantly reduce (up to $\approx 90\%$) the dataset size required to correctly generalize, further, there are memory loads that are prohibitive to Hebbian networks without sleeping (no matter the size and quality of the provided examples), but that are easily handled by the present "rested" neural networks.
翻译:培养人工神经网络所需的大量数据与生物对应方所需相对较少的数据之间的差距是机器学习的一个中心难题。 在这里,在生物信息处理的启发下,我们引入了一个通用的Hopfield网络,根据Hebb的在线学习处方建立神经元之间的双向联结,并允许(适当平流的)离线睡眠机制。此外,为了保留一个学习框架,我们不假定这里有模式,相反,我们让网络经历的只是由每个模式的吵闹例子抽样组成的数据集。我们用统计机械工具分析模型,并获得其作为控制参数功能功能的定量图像:由此形成的网络是一种关联记忆,用于通过离线睡眠学习在线范例、一般化和优化其储存能力。值得注意的是,睡眠机制总是显著地减少(最高为$\appolx 90 ⁇ $) 正确概括所需的数据集大小,此外,还有大量记忆负荷,其能力作为控制参数的功能,我们获得了定量图象:由此产生的网络是一种关联的记忆记忆,用于通过离线、一般和质量网络处理的模型,而无需沉睡,能够轻易地处理。