Deep-predictive-coding networks (DPCNs) are hierarchical, generative models. They rely on feed-forward and feed-back connections to modulate latent feature representations of stimuli in a dynamic and context-sensitive manner. A crucial element of DPCNs is a forward-backward inference procedure to uncover sparse, invariant features. However, this inference is a major computational bottleneck. It severely limits the network depth due to learning stagnation. Here, we prove why this bottleneck occurs. We then propose a new forward-inference strategy based on accelerated proximal gradients. This strategy has faster theoretical convergence guarantees than the one used for DPCNs. It overcomes learning stagnation. We also demonstrate that it permits constructing deep and wide predictive-coding networks. Such convolutional networks implement receptive fields that capture well the entire classes of objects on which the networks are trained. This improves the feature representations compared with our lab's previous non-convolutional and convolutional DPCNs. It yields unsupervised object recognition that surpass convolutional autoencoders and are on par with convolutional networks trained in a supervised manner.
翻译:深度预知编码网络(DPCN)是分级、基因模型,依靠进料和回反馈连接,以动态和背景敏感的方式调节刺激的潜在特征。DPCN的一个关键要素是发现稀有、不易变异特征的向后推推法程序。但是,这种推论是一个主要的计算瓶颈。由于学习停滞,这严重限制了网络的深度。在这里,我们证明为什么会出现这种瓶颈。然后,我们提出一个新的前推法战略,其基础是加速的近似梯度。这个战略的理论趋同保障比DPCN更快。它克服了学习停滞。我们还表明,它允许建立深而宽的预测-编码网络。这种卷动网络使用开放的字段,捕捉网络所训练的所有对象。这与我们实验室以前的非革命性和革命性变异性DPCN相比,改善了特征的表达方式。它产生了超越了革命性自动变异式网络的不超前对象识别方式,并且具有受监督的网络。