Concept bottleneck model (CBM) is a ubiquitous method that can interpret neural networks using concepts. In CBM, concepts are inserted between the output layer and the last intermediate layer as observable values. This helps in understanding the reason behind the outputs generated by the neural networks: the weights corresponding to the concepts from the last hidden layer to the output layer. However, it has not yet been possible to understand the behavior of the generalization error in CBM since a neural network is a singular statistical model in general. When the model is singular, a one to one map from the parameters to probability distributions cannot be created. This non-identifiability makes it difficult to analyze the generalization performance. In this study, we mathematically clarify the Bayesian generalization error and free energy of CBM when its architecture is three-layered linear neural networks. We also consider a multitask problem where the neural network outputs not only the original output but also the concepts. The results show that CBM drastically changes the behavior of the parameter region and the Bayesian generalization error in three-layered linear neural networks as compared with the standard version, whereas the multitask formulation does not.
翻译:概念瓶颈模型(CBM)是一种普遍的方法,可以使用概念来解释神经网络。在CBM中,概念被插入输出层和最后一个中间层之间作为可观测值。这有助于理解神经网络生成的输出背后的原因:来自最后一个隐藏层到输出层的与概念相对应的权重。然而,由于神经网络通常是奇异的统计模型,因此尚未能够理解CBM中的泛化误差行为。当模型是奇异的时候,参数到概率分布的一对一映射无法创建。这种不可识别性使得分析广义性能变得困难。在本研究中,我们在CBM的体系结构为三层线性神经网络的情况下,数学上澄清了CBM的贝叶斯广义误差和自由能。我们还考虑了一项多任务问题,其中神经网络不仅输出原始输出,还输出概念。结果表明,与标准版本相比,CBM在三层线性神经网络中极大地改变了参数区域和贝叶斯广义误差的行为,而多任务公式则没有。