Neural closure models have recently been proposed as a method for efficiently approximating small scales in multiscale systems with neural networks. The choice of loss function and associated training procedure has a large effect on the accuracy and stability of the resulting neural closure model. In this work, we systematically compare three distinct procedures: "derivative fitting", "trajectory fitting" with discretise-then-optimise, and "trajectory fitting" with optimise-then-discretise. Derivative fitting is conceptually the simplest and computationally the most efficient approach and is found to perform reasonably well on one of the test problems (Kuramoto-Sivashinsky) but poorly on the other (Burgers). Trajectory fitting is computationally more expensive but is more robust and is therefore the preferred approach. Of the two trajectory fitting procedures, the discretise-then-optimise approach produces more accurate models than the optimise-then-discretise approach. While the optimise-then-discretise approach can still produce accurate models, care must be taken in choosing the length of the trajectories used for training, in order to train the models on long-term behaviour while still producing reasonably accurate gradients during training. Two existing theorems are interpreted in a novel way that gives insight into the long-term accuracy of a neural closure model based on how accurate it is in the short term.
翻译:最近提出了神经封闭模型,作为在有神经网络的多规模系统中有效接近小型规模的方法。选择损失功能和相关培训程序对由此形成的神经封闭模型的准确性和稳定性产生很大影响。在这项工作中,我们系统地比较了三种不同的程序:“诊断性安装”、“轨迹安装”与离散-当时的优化相匹配,以及“轨迹安装”与优化-当时的分解。衍生匹配在概念上是最简单,也是计算上最有效的方法,在测试问题之一(Kuramoto-Sivashinsky)上表现得相当好,但在另一个测试问题(外科医生)上表现也很差。在计算性安装时,成本更昂贵,因此是首选的方法。在两个轨迹安装安装程序中,离散-现在的“轨道安装”方法比优化-当时的分解方法产生更准确的模型。虽然优化-当时的分解方法仍然能够产生准确的模型,并且发现在一个测试问题(Kuramotomoto-Sivashsky)上表现得相当好,但在选择长期的精确性精确度模型时,必须谨慎地在选择长期的精确的精确性模型中选择,在使用中进行长期的精确的精确的顺序中进行。