Comparing learned neural representations in neural networks is a challenging but important problem, which has been approached in different ways. The Centered Kernel Alignment (CKA) similarity metric, particularly its linear variant, has recently become a popular approach and has been widely used to compare representations of a network's different layers, of architecturally similar networks trained differently, or of models with different architectures trained on the same data. A wide variety of conclusions about similarity and dissimilarity of these various representations have been made using CKA. In this work we present analysis that formally characterizes CKA sensitivity to a large class of simple transformations, which can naturally occur in the context of modern machine learning. This provides a concrete explanation of CKA sensitivity to outliers, which has been observed in past works, and to transformations that preserve the linear separability of the data, an important generalization attribute. We empirically investigate several weaknesses of the CKA similarity metric, demonstrating situations in which it gives unexpected or counter-intuitive results. Finally we study approaches for modifying representations to maintain functional behaviour while changing the CKA value. Our results illustrate that, in many cases, the CKA value can be easily manipulated without substantial changes to the functional behaviour of the models, and call for caution when leveraging activation alignment metrics.
翻译:在神经网络中,比较所学的神经结构是一个具有挑战性但很重要的问题,这个问题已经以不同的方式处理。中央中枢对一大批简单转变的敏感性,特别是其线性变异,最近已成为一种流行的方法,并被广泛用来比较一个网络不同层、经过不同培训的建筑上相似的网络或由经过相同数据培训的不同结构组成的模型的表示;使用CKA对这些不同表述的相似性和不同性作出了各种各样的结论。在这项工作中,我们提出分析,正式地将CKA对一大批简单转变的敏感性定性为CKA的敏感性,这在现代机器学习中自然会发生。这具体解释了CKA对外部单位的敏感性,在过去的工程中已经观察到了这种敏感性,并具体地解释了保持数据线性分离性变化的转变,这是一个重要的概括性属性。我们用经验调查了CKA类似指标的一些弱点,表明它会产生出乎意料的或反直观的结果。我们最后研究的是,在改变CKA价值的同时,为了保持功能行为而改变CKA的敏感性,我们的结果表明,在许多情况下,在使功能性调整的调整成为基本调整时,我们的行为是谨慎的。我们的结果说明,在使CKA值的调整中,在不易动中可以要求使基本的调整中,我们的行动中,我们的结果表明,在不要求使驱动的调整的调整的调整中,在很多情况下,我们的行动是要求。