There has been growing interest in the AI community for precise uncertainty quantification. Conditional density models f(y|x), where x represents potentially high-dimensional features, are an integral part of uncertainty quantification in prediction and Bayesian inference. However, it is challenging to assess conditional density estimates and gain insight into modes of failure. While existing diagnostic tools can determine whether an approximated conditional density is compatible overall with a data sample, they lack a principled framework for identifying, locating, and interpreting the nature of statistically significant discrepancies over the entire feature space. In this paper, we present rigorous and easy-to-interpret diagnostics such as (i) the "Local Coverage Test" (LCT), which distinguishes an arbitrarily misspecified model from the true conditional density of the sample, and (ii) "Amortized Local P-P plots" (ALP) which can quickly provide interpretable graphical summaries of distributional differences at any location x in the feature space. Our validation procedures scale to high dimensions and can potentially adapt to any type of data at hand. We demonstrate the effectiveness of LCT and ALP through a simulated experiment and applications to prediction and parameter inference for image data.
翻译:AI社区对精确的不确定性量化越来越感兴趣。x代表着潜在的高维特征的有条件密度模型f(y ⁇ x)是预测和巴伊西亚推论中不确定性量化的一个有机组成部分,然而,评估有条件密度估计和深入了解失败模式是具有挑战性的。虽然现有的诊断工具可以确定一个大约的有条件密度总体上是否与数据样本相容,但它们缺乏一个原则性框架来查明、定位和解释整个地貌空间在统计上的重大差异的性质。在本文中,我们提出了严格和易于解释的诊断,例如(i) “局部覆盖测试”(LCT),该诊断将任意界定的模式与抽样的真正有条件密度区分开来,以及(ii) “局部P-P图示”(ALP),该分析工具能够迅速提供可解释的特征空间任何地点x的分布差异的图表摘要。我们的验证程序尺度达到高维度,并有可能适应手头的任何类型的数据。我们通过模拟实验和应用数据预测和参数,展示LCT和ALP的有效性。