Physics-informed neural networks (PINNs) numerically approximate the solution of a partial differential equation (PDE) by incorporating the residual of the PDE along with its initial/boundary conditions into the loss function. In spite of their partial success, PINNs are known to struggle even in simple cases where the closed-form analytical solution is available. In order to better understand the learning mechanism of PINNs, this work focuses on a systematic analysis of PINNs for the linear advection-diffusion equation (LAD) using the Neural Tangent Kernel (NTK) theory. Thanks to the NTK analysis, the effects of the advection speed/diffusion parameter on the training dynamics of PINNs are studied and clarified. We show that the training difficulty of PINNs is a result of 1) the so-called spectral bias, which leads to difficulty in learning high-frequency behaviours; and 2) convergence rate disparity between different loss components that results in training failure. The latter occurs even in the cases where the solution of the underlying PDE does not exhibit high-frequency behaviour. Furthermore, we observe that this training difficulty manifests itself, to some extent, differently in advection-dominated and diffusion-dominated regimes. Different strategies to address these issues are also discussed. In particular, it is demonstrated that periodic activation functions can be used to partly resolve the spectral bias issue.
翻译:物理知情神经网络(PINNs)通过将PDE的剩余部分及其初始/前沿条件纳入损失功能,从数字上接近部分差异方程式(PDE)的解决方案。尽管PINNs取得了部分成功,但据知即使在存在封闭式分析解决方案的简单情况下,PINNs也会挣扎。为了更好地了解PINNs的学习机制,这项工作侧重于利用Neal Tangent Kernel(NTK)理论系统分析线性平反扩散方程式(LAD),将部分差异方程式的剩余部分与部分差异方程式(PDE)相近。由于NTK(NTK)的分析,对PDE的倾销速度/扩散参数对PINNs培训动态的影响得到了研究和澄清。我们发现,PINNs的培训难度在于(1) 所谓的光谱偏差导致难以学习高频度行为;(2) 导致培训失败的不同损失方程式的趋同率差距。即使在基本PDE(NT)的解决方案不表现出高频度行为的情况下,也会出现这种情况。此外,我们注意到,这种培训的稳定性是典型的难题。