Functional Variational Inference based on Stochastic Process Generators

Bayesian inference in the space of functions has been an important topic for Bayesian modeling in the past. In this paper, we propose a new solution to this problem called Functional Variational Inference (FVI). In FVI, we minimize a divergence in function space between the variational distribution and the posterior process. This is done by using as functional variational family a new class of flexible distributions called Stochastic Process Generators (SPGs), which are cleverly designed so that the functional ELBO can be estimated efficiently using analytic solutions and mini-batch sampling. FVI can be applied to stochastic process priors when random function samples from those priors are available. Our experiments show that FVI consistently outperforms weight-space and function space VI methods on several tasks, which validates the effectiveness of our approach. 2 1 0 1 2 3 3 2 1 0 1 2 3 y(x( )) (a) functional BNN 3 2 1 0 1 2 3 3 2 1 0 1 2 3 y(x( )) (b) Mean field VI BNN