Neural networks are popular and useful in many fields, but they have the problem of giving high confidence responses for examples that are away from the training data. This makes the neural networks very confident in their prediction while making gross mistakes, thus limiting their reliability for safety-critical applications such as autonomous driving, space exploration, etc. This paper introduces a novel neuron generalization that has the standard dot-product-based neuron and the {\color{black} radial basis function (RBF)} neuron as two extreme cases of a shape parameter. Using a rectified linear unit (ReLU) as the activation function results in a novel neuron that has compact support, which means its output is zero outside a bounded domain. To address the difficulties in training the proposed neural network, it introduces a novel training method that takes a pretrained standard neural network that is fine-tuned while gradually increasing the shape parameter to the desired value. The theoretical findings of the paper are a bound on the gradient of the proposed neuron and a proof that a neural network with such neurons has the universal approximation property. This means that the network can approximate any continuous and integrable function with an arbitrary degree of accuracy. The experimental findings on standard benchmark datasets show that the proposed approach has smaller test errors than state-of-the-art competing methods and outperforms the competing methods in detecting out-of-distribution samples on two out of three datasets.
翻译:在许多领域,神经网络是受欢迎的,在许多领域是有用的,但是它们有对远离培训数据的例子给予高度信任的回答的问题。这使得神经网络在做出重大错误时,对预测非常有信心,从而限制其安全关键应用的可靠性,例如自主驱动、空间探索等。本文引入了一种新的神经神经常规化,它具有标准的点产品神经元和色{黑}弧基参数功能(RBF),作为两个极极端的形状参数。使用一个纠正的线性单元(RELU)作为激活功能的结果,在具有紧凑支持的新神经元中产生,这意味着其输出在封闭域之外为零。为了解决在培训拟议的神经网络方面遇到的困难,它引入了一种新颖的培训方法,该方法采用预先训练的标准神经网络,该神经神经神经元和色素参数逐渐提升到理想值。文件的理论结论与拟议的神经神经元参数的梯度有关,并证明神经元神经元神经元神经元神经元神经元神经元神经元神经元神经元神经元神经元的神经元网络具有普遍近值。这意味着网络可以近似性的新神经神经神经神经神经神经神经神经神经神经神经神经元的神经神经神经神经神经元的神经神经神经元的神经神经神经神经元的神经神经元的神经神经神经神经神经神经元的神经元的神经神经神经神经元的神经元的神经神经神经神经神经神经神经元的神经神经神经神经神经神经元的神经神经元的神经元的神经神经神经神经神经元的神经神经元的神经神经神经神经元的神经神经神经神经神经神经元的神经元的神经元的神经神经神经神经神经神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经神经神经神经元的神经元的神经元的神经元的神经神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的神经元的