Deep neural networks are powerful tools to detect hidden patterns in data and leverage them to make predictions, but they are not designed to understand uncertainty and estimate reliable probabilities. In particular, they tend to be overconfident. We address this problem by developing a novel training algorithm that can lead to more dependable uncertainty estimates, without sacrificing predictive power. The idea is to mitigate overconfidence by minimizing a loss function, inspired by advances in conformal inference, that quantifies model uncertainty by carefully leveraging hold-out data. Experiments with synthetic and real data demonstrate this method leads to smaller conformal prediction sets with higher conditional coverage, after exact calibration with hold-out data, compared to state-of-the-art alternatives.
翻译:深心神经网络是探测数据中隐藏模式并利用它们作出预测的有力工具,但它们的设计不是为了理解不确定性和估计可靠的概率,特别是,它们往往过于自信。我们通过开发一种新的培训算法来解决这个问题,这种算法可以导致更可靠的不确定性估计,同时又不牺牲预测力。 其想法是尽量减少一种损失功能,在一致性推论的进步的启发下,通过谨慎利用搁置数据来量化模型不确定性。 合成和真实数据的实验表明,这种方法导致在精确校准搁置数据之后,在与最先进的替代方法相比的情况下,在精确校准搁置数据之后,使用条件范围更高的符合的成套预测。