Object recognition is an important task for improving the ability of visual systems to perform complex scene understanding. Recently, the Exponential Linear Unit (ELU) has been proposed as a key component for managing bias shift in Convolutional Neural Networks (CNNs), but defines a parameter that must be set by hand. In this paper, we propose learning a parameterization of ELU in order to learn the proper activation shape at each layer in the CNNs. Our results on the MNIST, CIFAR-10/100 and ImageNet datasets using the NiN, Overfeat, All-CNN and ResNet networks indicate that our proposed Parametric ELU (PELU) has better performances than the non-parametric ELU. We have observed as much as a 7.28% relative error improvement on ImageNet with the NiN network, with only 0.0003% parameter increase. Our visual examination of the non-linear behaviors adopted by Vgg using PELU shows that the network took advantage of the added flexibility by learning different activations at different layers.
翻译:对象识别是提高视觉系统进行复杂场景理解的能力的一项重要任务。 最近, 已经提议将显微线性单元( ELU) 作为管理进化神经网络( CNNs) 中偏向转移的关键组成部分, 但定义了一个必须手工设定的参数 。 在本文中, 我们建议学习ELU 的参数化, 以便学习CNN每层的正常激活形状 。 我们对使用 NN、 overfeat、 All-CNN 和 ResNet 网络的 MNIST、 CIFAR- 10/100 和图像网数据集的研究结果显示, 我们拟议的参数ELU( PELU) 的性能优于非参数性能。 我们观察到, 与 NiN 网络相比, 图像网的相对差差幅提高了7. 28% 。 我们用 PELU 对 Vgg 采用的非线性行为进行的视觉检查显示, 网络通过在不同层次学习不同的激活手段, 利用 PELU 利用 PELU 来利用增加的灵活性。