In this paper, we show that the difference in Euclidean norm of samples can make a contribution to the semantic divergence and even confusion, after the spatial translation and scaling transformation in batch normalization. To address this issue, we propose an intuitive but effective method to equalize the Euclidean norms of sample vectors. Concretely, we $l_2$-normalize each sample vector before batch normalization, and therefore the sample vectors are of the same magnitude. Since the proposed method combines the $l_2$ normalization and batch normalization, we name our method as $L_2$BN. The $L_2$BN can strengthen the compactness of intra-class features and enlarge the discrepancy of inter-class features. In addition, it can help the gradient converge to a stable scale. The $L_2$BN is easy to implement and can exert its effect without any additional parameters and hyper-parameters. Therefore, it can be used as a basic normalization method for neural networks. We evaluate the effectiveness of $L_2$BN through extensive experiments with various models on image classification and acoustic scene classification tasks. The experimental results demonstrate that the $L_2$BN is able to boost the generalization ability of various neural network models and achieve considerable performance improvements.
翻译:在本文中,我们表明,在批量正常化的空间翻译和比例转换之后,欧洲的样本规范差异可以造成语义差异,甚至混乱。为了解决这一问题,我们建议一种直观但有效的方法,使欧洲的样本矢量规范均衡。具体地说,我们在批量正常化之前,将每个样本矢量统一起来,因此,样本矢量具有同等规模。由于拟议的方法将1美元2美元正常化和批量正常化结合起来,我们将我们的方法命名为2美元BN。$2BN可以加强类内特征的紧凑性,扩大类别间特征的差异。此外,它可以帮助梯度趋同到稳定的尺度。2BN美元易于执行,并且可以在不增加参数和超分度的情况下发挥作用。因此,它可以作为神经网络的基本正常化方法使用。我们通过在图像分类和声波级分类方面的各种模型进行的广泛实验,我们评估了$2美元BN的效益。 各种实验结果显示,能够使普通的推进力网络得到改进。