Reliable Probabilistic Forecasting of Irregular Time Series through Marginalization-Consistent Flows

Probabilistic forecasting models for joint distributions of targets in irregular time series with missing values are a heavily under-researched area in machine learning with, to the best of our knowledge, only two models are researched so far: the Gaussian Process Regression model (Dürichen et al., 2015) , and ProFITi (Yalavarthi et al., 2024b). While ProFITi, thanks to using multivariate normalizing flows, is very expressive, leading to better predictive performance, it suffers from marginalization inconsistency: it does not guarantee that the marginal distributions of a subset of variables in its predictive distributions coincide with the directly predicted distributions of these variables. When asked to directly predict marginal distributions, they are often vastly inaccurate. We propose MOSES (Marginalization Consistent Mixture of Separable Flows), a model that parametrizes a stochastic process through a mixture of several latent multivariate Gaussian Processes combined with separable univariate Normalizing Flows. In particular, MOSES can be analytically marginalized allowing it to directly answer a wider range of probabilistic queries than most competitors. Experiments on four datasets show that MOSES achieves both accurate joint and marginal predictions, surpassing all other marginalization consistent baselines, while only trailing slightly behind ProFITi in joint prediction, but vastly superior when predicting marginal distributions. probabilistic forecasting models for irregular time series (De Brouwer et al., 2019; Deng et al., 2020; Biloš et al., 2021; Schirmer et al., 2022) . However, these models typically focus on univariate forecasts at single time points. Yet many practical decisions ranging from diagnosing diseases to predicting weather depend on interactions between multiple variables over time, requiring accurate forecasts of joint multivariate distributions. This area remains underexplored, with only two notable models: Gaussian Process Regression (GPR) (Dürichen et al., 2015) , which models multivariate Gaussians, and ProFITi (Yalavarthi et al., 2024b), which uses normalizing flows for greater flexibility. ProFITi achieves stronger performance but lacks a key property: marginalization consistency which guarantees that marginal distributions are the same whether queried directly or derived from the joint. This consistency is crucial with varying numbers of observed variables. For instance, users ask a weather model for the probability of next three sunny days in San Diego and the chance of rain tomorrow. If the answers contradict each other, trust in the model erodes-even if prediction of three sunny days is accurate. In practice, we find that ProFITi, despite producing strong joint distributions, fails to maintain consistent marginals. On the other hand, GPR, while consistent, underperforms overall. From this starting point we constructed a novel model that combines the ideas of Gaussian Processes, normalizing flows and mixture models in a way completely different from ProFITi and GPR, to achieve both, guaranteed marginalization consistency and high predictive accuracy (see Figure 3 ). Overall our contributions as follows: 1. We formalize the underexplored property of marginalization consistency in probabilistic forecasting for irregular time series (Section 2). We propose Wasserstein Distance based metric to measure the marginalization inconsistency (Section 6.1). 2. We introduce a novel probabilistic forecasting model for irregular time series, Marginalization Consistent Mixtures of Separable Flows (MOSES). MOSES combines multiple normalizing flows with: (i) Gaussian Processes with full covariance matrices as source distributions (as opposed to the usual identity matrix), and (ii) a separable invertible transformation, applied independently per dimension rather than jointly. We formally prove that MOSES is guaranteed to be Marginalization Consistent (Sections 4 and 5). 3. In experiments on four datasets, we show that MOSES outperforms other state-of-theart marginalization-consistent models in both multivariate joint and univariate marginal distributions. While its performance on joint distributions is comparable to or slightly below that of ProFITi, MOSES significantly surpasses ProFITi in univariate marginals (Section 6), demonstrating the advantage of Marginalization Consistency. Code available at https://anonymous.4open.science/r/seperable_flows-BACC