This paper shows a Min-Max property existing in the connection weights of the convolutional layers in a neural network structure, i.e., the LeNet. Specifically, the Min-Max property means that, during the back propagation-based training for LeNet, the weights of the convolutional layers will become far away from their centers of intervals, i.e., decreasing to their minimum or increasing to their maximum. From the perspective of uncertainty, we demonstrate that the Min-Max property corresponds to minimizing the fuzziness of the model parameters through a simplified formulation of convolution. It is experimentally confirmed that the model with the Min-Max property has a stronger adversarial robustness, thus this property can be incorporated into the design of loss function. This paper points out a changing tendency of uncertainty in the convolutional layers of LeNet structure, and gives some insights to the interpretability of convolution.
翻译:本文显示了神经网络结构(即LeNet)中革命层的连接权重中存在的最小最大值属性。 具体地说, Min- Max属性意味着,在LeNet的后向传播培训中,革命层的重量将远离其周期中心,即降低到最小值或最大值。 从不确定性的角度来看,我们证明,最小值属性通过简化的演算公式来最大限度地减少模型参数的模糊性。它实验性地证实,与Min-Max属性有关的模型具有更强的对抗性坚固性,因此,这种属性可以被纳入损失功能的设计中。 本文指出了LeNet结构的革命层不断变化的不确定性趋势,并对演算的可解释性提供了一些见解。