Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filtering

Combinations of neural ODEs with recurrent neural networks (RNN), like GRUODE-Bayes or ODE-RNN are well suited to model irregularly-sampled time series. While those models outperform existing discrete-time approaches, no theoretical guarantees for their predictive capabilities are available. Assuming that the irregularly-sampled time series data originates from a continuous stochastic processes, the optimal on-line prediction is the conditional expectation given the currently available information. We introduce the Neural Jump ODE (NJ-ODE) that provides a data-driven approach to learn, continuously in time, the conditional expectation of a stochastic process. Our approach models the conditional expectation between two observations with a neural ODE and jumps whenever a new observation is made. We define a novel training framework, which allows us to prove theoretical convergence guarantees for the first time. In particular, we demonstrate the predictive capabilities of our model by proving that, under some regularity assumptions, the output process converges to the conditional expectation process. We provide experiments showing that the theoretical results also hold empirically. Moreover, we experimentally show that our model outperforms one state of the art model in more complex learning tasks and give comparisons on a real-world dataset.