Recent research has used deep learning to develop partial differential equation (PDE) models in science and engineering. The functional form of the PDE is determined by a neural network, and the neural network parameters are calibrated to available data. Calibration of the embedded neural network can be performed by optimizing over the PDE. Motivated by these applications, we rigorously study the optimization of a class of linear elliptic PDEs with neural network terms. The neural network parameters in the PDE are optimized using gradient descent, where the gradient is evaluated using an adjoint PDE. As the number of parameters become large, the PDE and adjoint PDE converge to a non-local PDE system. Using this limit PDE system, we are able to prove convergence of the neural network-PDE to a global minimum during the optimization. The limit PDE system contains a non-local linear operator whose eigenvalues are positive but become arbitrarily small. The lack of a spectral gap for the eigenvalues poses the main challenge for the global convergence proof. Careful analysis of the spectral decomposition of the coupled PDE and adjoint PDE system is required. Finally, we use this adjoint method to train a neural network model for an application in fluid mechanics, in which the neural network functions as a closure model for the Reynolds-averaged Navier-Stokes (RANS) equations. The RANS neural network model is trained on several datasets for turbulent channel flow and is evaluated out-of-sample at different Reynolds numbers.
翻译:最近的研究利用了深层的学习来开发科学和工程方面的部分差异方程(PDE)模型。PDE的功能形式由神经网络确定,神经网络参数则根据可用数据校准。对嵌入神经网络的校准可以通过优化 PDE 进行。受这些应用的激励,我们严格研究以神经网络条件优化一类线性椭圆形PDE 。PDE 的神经网络参数使用梯度下沉来优化,梯度使用一个联合PDE来评估。随着参数数目的增多,PDE 和联合PDE 的功能形式会与非本地的 PDE 系统趋同。利用这一限制,我们可以证明内嵌神经网络-PDE 网络与全球最小值的趋同。限制PDE 系统包含一个非本地线性线性操作器,其树皮值是正值,但变得非常小。对于梯值的光谱差距是全球趋同证据的主要挑战。仔细分析相光线性内线性内径朗网络的线性内径阵列流流数据,最后需要用这个模型来进行内存的内置的内存的内存的内径径性内径里亚-内径性网络的内径径径径径径径径径流数据系统。