Understanding the functional principles of information processing in deep neural networks continues to be a challenge, in particular for networks with trained and thus non-random weights. To address this issue, we study the mapping between probability distributions implemented by a deep feed-forward network. We characterize this mapping as an iterated transformation of distributions, where the non-linearity in each layer transfers information between different orders of correlation functions. This allows us to identify essential statistics in the data, as well as different information representations that can be used by neural networks. Applied to an XOR task and to MNIST, we show that correlations up to second order predominantly capture the information processing in the internal layers, while the input layer also extracts higher-order correlations from the data. This analysis provides a quantitative and explainable perspective on classification.
翻译:了解深神经网络信息处理的功能性原则仍然是一项挑战,特别是对于受过训练的网络来说,这仍然是一项挑战,特别是对于具有非随机加权的网络来说。为了解决这一问题,我们研究了深线进料前网络所实施的概率分布图。我们将这一映射图描述为分布的迭代变化,在分布图中,每一层的非线性在不同相关功能的顺序之间传递信息。这使我们能够确定数据中的基本统计数据,以及神经网络可以使用的不同信息表述。应用到 XOR 任务和 MNIST,我们发现,到第二顺序的关联主要是内部层的信息处理,而输入层也从数据中提取了较高顺序的关联。这一分析为分类提供了定量和可解释的视角。