This paper investigates the usage of kernel functions at the different layers in a convolutional neural network. We carry out extensive studies of their impact on convolutional, pooling and fully-connected layers. We notice that the linear kernel may not be sufficiently effective to fit the input data distributions, whereas high order kernels prone to over-fitting. This leads to conclude that a trade-off between complexity and performance should be reached. We show how one can effectively leverage kernel functions, by introducing a more distortion aware pooling layers which reduces over-fitting while keeping track of the majority of the information fed into subsequent layers. We further propose Kernelized Dense Layers (KDL), which replace fully-connected layers, and capture higher order feature interactions. The experiments on conventional classification datasets i.e. MNIST, FASHION-MNIST and CIFAR-10, show that the proposed techniques improve the performance of the network compared to classical convolution, pooling and fully connected layers. Moreover, experiments on fine-grained classification i.e. facial expression databases, namely RAF-DB, FER2013 and ExpW demonstrate that the discriminative power of the network is boosted, since the proposed techniques improve the awareness to slight visual details and allows the network reaching state-of-the-art results.
翻译:本文调查了在进化神经网络中不同层次使用内核函数的情况。 我们对内核函数对进化、集中和完全连接层的影响进行了广泛研究。我们注意到线性内核可能不够有效,无法适应输入数据分布,而高排序内核容易过度配置。这导致得出复杂与性能之间的权衡应当达到。我们展示了如何通过引入更扭曲的认知集合层来有效地利用内核功能,从而减少过度配置,同时跟踪输入到随后层的大多数信息。我们进一步提议使用内核化的登塞层(KDL),以取代完全连接层,并捕捉更高顺序特征的相互作用。关于传统分类数据集的实验,即MNIST、FASHION-MNIST和CIFAR-10,表明拟议的技术将改进网络的性能与古典演化、联合和完全连接层相比。此外,关于美化表达数据库的实验,即RAF-DB、FER2013和ExplainW, 显示拟议的网络的微量能提升了自拟议的网络以来的视觉结果。