Variational Inference for Sequential Data with Future Likelihood Estimates

The recent development of flexible and scalable variational inference algorithms has popularized the use of deep probabilistic models in a wide range of applications. However, learning and rea soning about high-dimensional models with nondifferentiable densities are still a challenge. For such a model, inference algorithms struggle to estimate the gradients of variational objectives ac curately, due to high variance in their estimates. To tackle this challenge, we present a novel vari ational inference algorithm for sequential data, which performs well even when the density from the model is not differentiable, for instance, due to the use of discrete random variables. The key feature of our algorithm is that it estimates future likelihoods at all time steps. The estimated future likelihoods form the core of our new low-variance gradient estimator. We formally analyze our gra dient estimator from the perspective of variational objective, and show the effectiveness of our algo rithm with synthetic and real datasets.