The past decade has seen increasing interest in applying Deep Learning (DL) to Computational Science and Engineering (CSE). Driven by impressive results in applications such as computer vision, Uncertainty Quantification (UQ), genetics, simulations and image processing, DL is increasingly supplanting classical algorithms, and seems poised to revolutionize scientific computing. However, DL is not yet well-understood from the standpoint of numerical analysis. Little is known about the efficiency and reliability of DL from the perspectives of stability, robustness, accuracy, and sample complexity. In particular, approximating solutions to parametric PDEs is an objective of UQ for CSE. Training data for such problems is often scarce and corrupted by errors. Moreover, the target function is a possibly infinite-dimensional smooth function taking values in the PDE solution space, generally an infinite-dimensional Banach space. This paper provides arguments for Deep Neural Network (DNN) approximation of such functions, with both known and unknown parametric dependence, that overcome the curse of dimensionality. We establish practical existence theorems that describe classes of DNNs with dimension-independent architecture size and training procedures based on minimizing the (regularized) $\ell^2$-loss which achieve near-optimal algebraic rates of convergence. These results involve key extensions of compressed sensing for Banach-valued recovery and polynomial emulation with DNNs. When approximating solutions of parametric PDEs, our results account for all sources of error, i.e., sampling, optimization, approximation and physical discretization, and allow for training high-fidelity DNN approximations from coarse-grained sample data. Our theoretical results fall into the category of non-intrusive methods, providing a theoretical alternative to classical methods for high-dimensional approximation.
翻译:过去十年中,人们越来越有兴趣将Deep Learning(DL)应用到计算科学和工程(CSE) 。在计算机视觉、不确定的定量(UQ)、遗传学、模拟和图像处理等应用的令人印象深刻的结果的驱使下,DL正在日益取代古典算法,并似乎准备将科学计算革命化。然而,从数字分析的角度来看,DL还没有被很好地理解。从稳定性、稳健性、准确性和抽样复杂性的角度来看,DL的效能和可靠性鲜为人知。特别是,对参数PDE的接近性解决方案是 CSEQ的目标。 这些问题的培训数据往往很少,而且由于错误而腐蚀。 此外,DLL的目标功能可能是一个无限的光滑功能,在PDE解决方案空间中,一般是一个无限的Banach空间。 本文为深神经网络(DNNN)这类功能的近似近似性、已知和未知的偏差依赖性依赖性,从而克服了维度的诅咒。我们用实际的存在时间点定位数据流数据流流流化的恢复数据流化数据流化数据流化数据流化数据,用来描述DNA的正统化的系统,从而实现了DNA的正统化数据结构的升级的系统,从而实现了我们的直位化数据流化。