Neural saturation in Deep Neural Networks (DNNs) has been studied extensively, but remains relatively unexplored in Convolutional Neural Networks (CNNs). Understanding and alleviating the effects of convolutional kernel saturation is critical for enhancing CNN models classification accuracies. In this paper, we analyze the effect of convolutional kernel saturation in CNNs and propose a simple data augmentation technique to mitigate saturation and increase classification accuracy, by supplementing negative images to the training dataset. We hypothesize that greater semantic feature information can be extracted using negative images since they have the same structural information as standard images but differ in their data representations. Varied data representations decrease the probability of kernel saturation and thus increase the effectiveness of kernel weight updates. The two datasets selected to evaluate our hypothesis were CIFAR- 10 and STL-10 as they have similar image classes but differ in image resolutions thus making for a better understanding of the saturation phenomenon. MNIST dataset was used to highlight the ineffectiveness of the technique for linearly separable data. The ResNet CNN architecture was chosen since the skip connections in the network ensure the most important features contributing the most to classification accuracy are retained. Our results show that CNNs are indeed susceptible to convolutional kernel saturation and that supplementing negative images to the training dataset can offer a statistically significant increase in classification accuracies when compared against models trained on the original datasets. Our results present accuracy increases of 6.98% and 3.16% on the STL-10 and CIFAR-10 datasets respectively.
翻译:深神经网络(DNN) 的神经饱和度得到了广泛的研究,但在进化神经网络(CNNs)中仍然相对没有探索,但是在进化神经网络(CNNs)中仍然相对没有探索更多的语义特征信息。理解并减轻进化内核饱和效应对于加强CNN模型分类理解和减轻进化内核饱和效应至关重要。在本文中,我们分析了CNN的进化内核饱和效应的影响,并提出了一个简单的数据增强技术,通过对培训数据集的负图像进行补充来减轻饱和增加分类准确性。我们假设中更多的语义特征信息可以用负面图像来提取,因为它们具有与标准图像相同的结构信息。理解并显示进化内核内核饱饱和效应的效果。在经过培训的原始模型中,可以突出直线性精确度技术的有效性,而在经过训练的图像中,我们所选择的内核电解内核电解内核数据结构可以分别提供重要的数据。