Probabilistic predictions from neural networks which account for predictive uncertainty during classification is crucial in many real-world and high-impact decision making settings. However, in practice most datasets are trained on non-probabilistic neural networks which by default do not capture this inherent uncertainty. This well-known problem has led to the development of post-hoc calibration procedures, such as Platt scaling (logistic), isotonic and beta calibration, which transforms the scores into well calibrated empirical probabilities. A plausible alternative to the calibration approach is to use Bayesian neural networks, which directly models a predictive distribution. Although they have been applied to images and text datasets, they have seen limited adoption in the tabular and small data regime. In this paper, we demonstrate that Bayesian neural networks yields competitive performance when compared to calibrated neural networks and conduct experiments across a wide array of datasets.
翻译:在许多现实世界和高影响的决策环境中,分类过程中预测不确定性的神经网络的概率预测至关重要,然而,在实践中,大多数数据集都接受非概率神经网络的培训,但默认情况下无法捕捉这种内在不确定性。这一众所周知的问题导致热后校准程序的发展,如Platt定级(物流)、异调和贝塔校准,从而将评分转换成经充分校准的经验概率。校准方法的一个合理替代办法是使用Bayesian神经网络,直接模拟预测分布。尽管这些数据集已被应用到图像和文本数据集中,但它们在表格和小数据系统中被采用的程度有限。在本文中,我们证明Bayesian神经网络与校准神经网络相比产生有竞争力的性能,并在一系列广泛的数据集中进行实验。