We focus on learning unknown dynamics from data using ODE-nets templated on implicit numerical initial value problem solvers. First, we perform Inverse Modified error analysis of the ODE-nets using unrolled implicit schemes for ease of interpretation. It is shown that training an ODE-net using an unrolled implicit scheme returns a close approximation of an Inverse Modified Differential Equation (IMDE). In addition, we establish a theoretical basis for hyper-parameter selection when training such ODE-nets, whereas current strategies usually treat numerical integration of ODE-nets as a black box. We thus formulate an adaptive algorithm which monitors the level of error and adapts the number of (unrolled) implicit solution iterations during the training process, so that the error of the unrolled approximation is less than the current learning loss. This helps accelerate training, while maintaining accuracy. Several numerical experiments are performed to demonstrate the advantages of the proposed algorithm compared to nonadaptive unrollings, and validate the theoretical analysis. We also note that this approach naturally allows for incorporating partially known physical terms in the equations, giving rise to what is termed ``gray box" identification.
翻译:我们着眼于使用以隐式数值初值问题求解程序为模板的ODE-net来从数值数据中学习未知的动态过程。首先,我们使用展开的隐式方案执行ODE-net的反向修改误差分析以便更容易理解。研究证明,使用展开的隐式方案训练ODE-net返回接近反向修改微分方程 (IMDE) 的近似值。此外,我们建立了训练此类ODE-net时设置超参数的理论基础,而现有的策略通常将ODE-net的数值积分视为一个黑盒子。因此,我们提出了一种自适应算法,通过监控误差水平并在训练过程中调整(展开的)隐式解决方案的迭代次数,使得展开的近似误差小于当前的学习损失。这有助于加速训练,同时维持准确性。我们进行了几个数字实验来展示所提出算法与非自适应展开之间的优势,并验证了理论分析。我们还指出,此方法自然地允许将部分已知的物理项纳入方程中,从而产生所谓的“灰盒子”识别过程。