Although Physics-Informed Neural Networks (PINNs) have been successfully applied in a wide variety of science and engineering fields, they can fail to accurately predict the underlying solution in slightly challenging convection-diffusion-reaction problems. In this paper, we investigate the reason of this failure from a domain distribution perspective, and identify that learning multi-scale fields simultaneously makes the network unable to advance its training and easily get stuck in poor local minima. We show that the widespread experience of sampling more collocation points in high-loss layer regions hardly help optimize and may even worsen the results. These findings motivate the development of a novel curriculum learning method that encourages neural networks to prioritize learning on easier non-layer regions while downplaying learning on harder layer regions. The proposed method helps PINNs automatically adjust the learning emphasis and thereby facilitate the optimization procedure. Numerical results on typical benchmark equations show that the proposed curriculum learning approach mitigates the failure modes of PINNs and can produce accurate results for very sharp boundary and interior layers. Our work reveals that for equations whose solutions have large scale differences, paying less attention to high-loss regions can be an effective strategy for learning them accurately.
翻译:尽管物理结构化神经网络(PINNs)在广泛的科学和工程领域中被成功应用,但在稍微具有挑战性的奇异扰动对流扩散反应问题中,它们可能无法准确预测潜在解。在本文中,我们从域分布的角度研究该失败的原因,并确定学习多尺度场同时使网络无法推进其训练并容易陷入较差的局部极小值。我们表明,采样更多高损耗层区域的经验很少有助于优化,甚至可能加剧结果。这些发现激发了一种新的课程学习方法的发展,该方法鼓励神经网络优先学习较容易的非层区域,同时降低对较困难的层区域的学习。所提出的方法帮助PINNs自动调整学习重点,从而促进了优化过程。典型基准方程的数值结果表明,所提出的课程学习方法缓解了PINNs的失败模式,并且可以产生对于非常尖锐的边界和内部层的准确结果。我们的工作揭示了,对于解具有大尺度差异的方程,降低对高损失区域的关注可能是一种准确学习的有效策略。