This paper reviews predictive coding, from theoretical neuroscience, and variational autoencoders, from machine learning, identifying the common origin and mathematical framework underlying both areas. As each area is prominent within its respective field, more firmly connecting these areas could prove useful in the dialogue between neuroscience and machine learning. After reviewing each area, we discuss two possible correspondences implied by this perspective: cortical pyramidal dendrites as analogous to (non-linear) deep networks and lateral inhibition as analogous to normalizing flows. These connections may provide new directions for further investigations in each field.
翻译:本文回顾了从理论神经科学和变异自动编码器、从机器学习得出的预测编码,确定了这两个领域的共同起源和数学框架。由于每个领域在各自领域都很突出,因此更牢固地连接这些领域可能有助于神经科学和机器学习之间的对话。在审查了每个领域之后,我们讨论了这一视角所隐含的两种可能的对应关系:皮层金字塔式下层类似于(非线性)深层网络,横向抑制类似于正常流动。这些连接可为每个领域的进一步调查提供新的方向。