Neural networks are popular and useful in many fields, but they have the problem of giving high confidence responses for examples that are away from the training data. This makes the neural networks very confident in their prediction while making gross mistakes, thus limiting their reliability for safety-critical applications such as autonomous driving, space exploration, etc. In this paper, we present a neuron generalization that has the standard dot-product-based neuron and the RBF neuron as two extreme cases of a shape parameter. Using ReLU as the activation function we obtain a novel neuron that has compact support, which means its output is zero outside a bounded domain. We show how to avoid difficulties in training a neural network with such neurons, by starting with a trained standard neural network and gradually increasing the shape parameter to the desired value. Through experiments on standard benchmark datasets, we show the promise of the proposed approach, in that it can have good prediction accuracy on in-distribution samples while being able to consistently detect and have low confidence on out-of-distribution samples.
翻译:神经网络在很多领域都很受欢迎,也很有用,但是它们存在对远离培训数据的例子给予高度自信反应的问题。这使得神经网络在做出重大错误的同时,对预测非常有信心,从而限制了其安全关键应用的可靠性,如自主驱动、空间探索等。 在本文中,我们提出了一个神经系统一般化,将标准点产品神经元和RBF神经元作为两种极端的形状参数。使用RELU作为激活功能,我们获得了一种具有紧凑支持的新型神经元,这意味着其输出在封闭域之外是零。我们展示了如何避免在培训神经神经网络方面遇到困难,先从训练有素的标准神经网络开始,逐步提高形状参数,使其达到预期值。我们通过标准基准数据集实验,展示了拟议方法的许诺,即它能够很好地预测分配样本的准确性,同时能够持续地检测和低地信任分配之外的样本。