Deep neural network (DNN) models have achieved state-of-the-art predictive accuracy in a wide range of supervised learning applications. However, accurately quantifying the uncertainty in DNN predictions remains a challenging task. For continuous outcome variables, an even more difficult problem is to estimate the predictive density function, which not only provides a natural quantification of the predictive uncertainty, but also fully captures the random variation in the outcome. In this work, we propose the Bayesian Deep Noise Neural Network (B-DeepNoise), which generalizes standard Bayesian DNNs by extending the random noise variable from the output layer to all hidden layers. The latent random noise equips B-DeepNoise with the flexibility to approximate highly complex predictive distributions and accurately quantify predictive uncertainty. For posterior computation, the unique structure of B-DeepNoise leads to a closed-form Gibbs sampling algorithm that iteratively simulates from the posterior full conditional distributions of the model parameters, circumventing computationally intensive Metropolis-Hastings methods. A theoretical analysis of B-DeepNoise establishes a recursive representation of the predictive distribution and decomposes the predictive variance with respect to the latent parameters. We evaluate B-DeepNoise against existing methods on benchmark regression datasets, demonstrating its superior performance in terms of prediction accuracy, uncertainty quantification accuracy, and uncertainty quantification efficiency. To illustrate our method's usefulness in scientific studies, we apply B-DeepNoise to predict general intelligence from neuroimaging features in the Adolescent Brain Cognitive Development (ABCD) project.
翻译:深神经网络(DNN) 模型在广泛监督的学习应用中达到了最先进的预测准确性。 但是,准确量化DNN预测中的不确定性仍然是一项艰巨的任务。 对于持续的结果变量来说,更困难的问题是估算预测密度功能,它不仅为预测不确定性提供了自然量化,而且充分捕捉了结果的随机变化。在这项工作中,我们提议Bayesian深噪音神经网络(B-深噪音网络),它通过将随机的噪音变异从输出层扩大到所有隐藏层,将标准的Bayesian DNNNNS普遍化。潜伏随机噪音使B-DepNoise具备了灵活性,可以接近高度复杂的预测分布和准确量化预测不确定性。对于海景计算来说,B-Dep Novise的独特结构导致一个闭板式的抽样算法,从模型参数的事后完全有条件分布中反复模拟,绕过计算密集的Metopoli-Hasing方法。 B-D-D-Nismidality的理论分析,从B-D-D-Nision Recialimimal Studyal 研究到我们现有的预测方法的精确性分析。