A neural network is modular to the extent that parts of its computational graph (i.e. structure) can be represented as performing some comprehensible subtask relevant to the overall task (i.e. functionality). Are modern deep neural networks modular? How can this be quantified? In this paper, we consider the problem of assessing the modularity exhibited by a partitioning of a network's neurons. We propose two proxies for this: importance, which reflects how crucial sets of neurons are to network performance; and coherence, which reflects how consistently their neurons associate with features of the inputs. To measure these proxies, we develop a set of statistical methods based on techniques conventionally used to interpret individual neurons. We apply the proxies to partitionings generated by spectrally clustering a graph representation of the network's neurons with edges determined either by network weights or correlations of activations. We show that these partitionings, even ones based only on weights (i.e. strictly from non-runtime analysis), reveal groups of neurons that are important and coherent. These results suggest that graph-based partitioning can reveal modularity and help us understand how deep neural networks function.
翻译:神经网络是模块化的, 其计算图( 即结构) 的某些部分可以表现为执行与总体任务( 功能) 相关的可理解子任务( 即结构) 。 现代深神经网络是模块吗? 如何量化? 在本文件中, 我们考虑评估通过网络神经元分割显示的模块化问题。 我们为此建议了两个替代点: 重要性, 它反映了神经元对网络性能的关键组合; 一致性, 它反映了它们的神经元与投入特征之间的持续联系。 为了测量这些代理点, 我们根据传统用来解释单个神经元的技术, 开发了一套统计方法。 我们用光谱组合生成的模型来分割网络神经元, 其边缘由网络重量或激活的关联决定。 我们显示这些分区, 甚至是仅仅基于重量( 严格来自非运行时间分析), 揭示了重要和连贯的神经元组。 这些结果表明, 基于图形分割的网络能够显示我们如何理解深度的模块化功能。