Combinations of neural ODEs with recurrent neural networks (RNN), like GRU-ODE-Bayes or ODE-RNN are well suited to model irregularly observed time series. While those models outperform existing discrete-time approaches, no theoretical guarantees for their predictive capabilities are available. Assuming that the irregularly-sampled time series data originates from a continuous stochastic process, the $L^2$-optimal online prediction is the conditional expectation given the currently available information. We introduce the Neural Jump ODE (NJ-ODE) that provides a data-driven approach to learn, continuously in time, the conditional expectation of a stochastic process. Our approach models the conditional expectation between two observations with a neural ODE and jumps whenever a new observation is made. We define a novel training framework, which allows us to prove theoretical guarantees for the first time. In particular, we show that the output of our model converges to the $L^2$-optimal prediction. This can be interpreted as solution to a special filtering problem. We provide experiments showing that the theoretical results also hold empirically. Moreover, we experimentally show that our model outperforms the baselines in more complex learning tasks and give comparisons on real-world datasets.
翻译:神经元与恒定神经网络(RNN)(RNN)的合并,如GRU-ODE-Bayes或OD-RNN,完全适合模拟不规则观测的时间序列。这些模型优于现有的离散时间方法,但无法为预测能力提供理论保证。假设不规则复制的时间序列数据来自连续的随机过程,$L2美元的最佳在线预测是目前可获得的信息中有条件的预期。我们引入了神经跳跃 ODE(NJ-OD),提供数据驱动方法,以持续地及时了解对随机过程的有条件期望。我们的方法模型是两种带有神经值的观测之间的有条件期望,每当出现新的观测时,我们就会跳跃。我们定义了一个新的培训框架,使我们能够证明第一次的理论保证。特别是,我们显示我们的模型输出结果与$L2美元-Oppyimal 预测(NJJ-ODE)相匹配,这可以被解释为一种特殊的筛选问题的解决办法。我们提供实验性实验,显示理论结果也显示复杂的实验性世界数据。此外,我们提供了一种实验性的数据。