Quantification of uncertainty is one of the most promising approaches to establish safe machine learning. Despite its importance, it is far from being generally solved, especially for neural networks. One of the most commonly used approaches so far is Monte Carlo dropout, which is computationally cheap and easy to apply in practice. However, it can underestimate the uncertainty. We propose a new objective, referred to as second-moment loss (SML), to address this issue. While the full network is encouraged to model the mean, the dropout networks are explicitly used to optimize the model variance. We intensively study the performance of the new objective on various UCI regression datasets. Comparing to the state-of-the-art of deep ensembles, SML leads to comparable prediction accuracies and uncertainty estimates while only requiring a single model. Under distribution shift, we observe moderate improvements. As a side result, we introduce an intuitive Wasserstein distance-based uncertainty measure that is non-saturating and thus allows to resolve quality differences between any two uncertainty estimates.
翻译:不确定性的量化是建立安全机器学习的最有希望的方法之一。 尽管它很重要,但它远未普遍解决,特别是神经网络。迄今为止最常用的方法之一是蒙特卡洛辍学,这是计算成本低且在实践中容易应用的。然而,它可能低估不确定性。我们提出了一个新的目标,称为第二步损失(SML),以解决这一问题。虽然鼓励整个网络模拟这一平均值,但辍学网络被明确用于优化模型差异。我们深入研究了各种UCI回归数据集的新目标的绩效。与深层集合的最新水平相比,SML导致可比较的预测理解性和不确定性估计数,而只需要一个单一的模式。在分配变化中,我们观察到适度的改进。作为附带结果,我们引入了一种不饱和的直觉瓦塞鲁斯坦远程不确定性测量,从而能够解决任何两种不确定性估算的质量差异。