Understanding the operation of biological and artificial networks remains a difficult and important challenge. To identify general principles, researchers are increasingly interested in surveying large collections of networks that are trained on, or biologically adapted to, similar tasks. A standardized set of analysis tools is now needed to identify how network-level covariates -- such as architecture, anatomical brain region, and model organism -- impact neural representations (hidden layer activations). Here, we provide a rigorous foundation for these analyses by defining a broad family of metric spaces that quantify representational dissimilarity. Using this framework we modify existing representational similarity measures based on canonical correlation analysis to satisfy the triangle inequality, formulate a novel metric that respects the inductive biases in convolutional layers, and identify approximate Euclidean embeddings that enable network representations to be incorporated into essentially any off-the-shelf machine learning method. We demonstrate these methods on large-scale datasets from biology (Allen Institute Brain Observatory) and deep learning (NAS-Bench-101). In doing so, we identify relationships between neural representations that are interpretable in terms of anatomical features and model performance.
翻译:了解生物和人工网络的运作仍然是一项困难和重要的挑战。为了确定一般原则,研究人员越来越有兴趣调查大量关于类似任务的培训或生物适应的网络,现在需要一套标准化的分析工具,以确定网络一级的共变(如建筑、解剖脑区域和模型生物)影响神经表象(隐形层激活)如何进行。在这里,我们通过界定一个能量化代表性差异的宽广的计量空间体系,为这些分析提供了严格的基础。我们利用这个框架,根据对三角间差异的可理解性相关分析,修改现有的代表性相似性措施,制定一套新颖的衡量标准,尊重卷发层的内含偏见,并查明大约的Euclidean嵌入,使网络表象基本上被纳入任何现成的机器学习方法。我们用生物(Allen研究所脑观测台)和深层学习(NAS-Bench-101)的大规模数据集展示了这些方法。我们这样做是为了查明在解剖特征和模型方面可以解释的神经表象之间的关系。