Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. The typical approach is to incorporate physical domain knowledge as soft constraints on an empirical loss function and use existing machine learning methodologies to train the model. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs. In particular, we analyze several distinct situations of widespread physical interest, including learning differential equations with convection, reaction, and diffusion operators. We provide evidence that the soft regularization in PINNs, which involves differential operators, can introduce a number of subtle problems, including making the problem ill-conditioned. Importantly, we show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize. We then describe two promising solutions to address these failure modes. The first approach is to use curriculum regularization, where the PINN's loss term starts from a simple PDE regularization, and becomes progressively more complex as the NN gets trained. The second approach is to pose the problem as a sequence-to-sequence learning task, rather than learning to predict the entire space-time at once. Extensive testing shows that we can achieve up to 1-2 orders of magnitude lower error with these methods as compared to regular PINN training.
翻译:科学机器学习的近期工作发展了所谓的物理知情神经网络(PINN)模型。典型的方法是将物理域知识作为实验性损失功能的软约束纳入物理域知识,将物理域知识作为实验性损失功能的软约束,并使用现有的机器学习方法来培训模型。我们证明,虽然现有的PINN方法可以针对相对微不足道的问题学习好模式,但即使对于简单的PDE,它们也很容易地无法了解相关的物理现象。特别是,我们分析一些具有广泛物理意义的不同情况,包括学习与对流、反应和传播操作者的不同方程式。我们提供了证据,证明PINNN的软性规范化涉及不同的操作者,可以引入一些微妙的问题,包括使问题条件不完善。我们表明,这些可能的失败模式并不是因为NN结构缺乏清晰度,但是PIN的设置使得损失场景很难优化。我们然后描述解决这些失败模式的两种有希望的解决办法。我们的第一个办法是使用课程规范,PINNN的损失术语始于简单的PDE正规化,并且随着NN的常规化而变得日益复杂,因为相对于NNN的常规化,我们所训练的深度的顺序可以显示一个更深层次的顺序。我们所要达到的深层次。我们所要完成的任务的进度。我们所要达到的深度的顺序,第二个的方法是要显示的深度的测测测测测测。我们所要达到一个测的层次。