Neural networks (NNs) are currently changing the computational paradigm on how to combine data with mathematical laws in physics and engineering in a profound way, tackling challenging inverse and ill-posed problems not solvable with traditional methods. However, quantifying errors and uncertainties in NN-based inference is more complicated than in traditional methods. This is because in addition to aleatoric uncertainty associated with noisy data, there is also uncertainty due to limited data, but also due to NN hyperparameters, overparametrization, optimization and sampling errors as well as model misspecification. Although there are some recent works on uncertainty quantification (UQ) in NNs, there is no systematic investigation of suitable methods towards quantifying the total uncertainty effectively and efficiently even for function approximation, and there is even less work on solving partial differential equations and learning operator mappings between infinite-dimensional function spaces using NNs. In this work, we present a comprehensive framework that includes uncertainty modeling, new and existing solution methods, as well as evaluation metrics and post-hoc improvement approaches. To demonstrate the applicability and reliability of our framework, we present an extensive comparative study in which various methods are tested on prototype problems, including problems with mixed input-output data, and stochastic problems in high dimensions. In the Appendix, we include a comprehensive description of all the UQ methods employed, which we will make available as open-source library of all codes included in this framework.
翻译:神经网络(NNs)目前正在改变关于如何将数据与物理和工程数学法相结合的计算模式,以深刻的方式处理传统方法无法解决的具有挑战性的反向和错误的问题。然而,对基于NN的推论中的错误和不确定性进行量化比传统方法更为复杂。这是因为,除了与噪音数据相关的偏移不确定性外,由于数据有限,还存在不确定性,但也由于NN的超参数、过度平衡、优化和抽样错误以及模型的不精确性。虽然最近就NNPs的不确定性量化(UQ)开展了一些工作,但没有系统地调查如何采用适当方法,以有效和高效地量化整个不确定性(即使是功能近似),而解决部分差异方程式和学习操作者绘制使用NNPs的无限维功能空间之间的图的工作甚至更少。在这项工作中,我们提出了一个综合框架,其中包括不确定性的模型、新的和现有的解决方案方法,以及评估指标和后改进方法。为了证明我们框架的不确定性的可适用性和可靠性,我们提出一个广泛的模型,我们用的方法中包括了我们所使用的高层次的模型。