Bridging the Gap Between f-divergences and Bayes Hilbert Spaces

We introduce a novel framework that generalizes f -divergences by incorporating divergence-generating functions that exhibit non-convex behavior in a neighborhood of the origin. Using this extension, we define a new class of pseudo fdivergences, encompassing a wider range of distances between distributions that traditional f -divergences cannot capture. Among these, we focus on a particular pseudo divergence, obtained by considering the induced metric of Bayes Hilbert spaces. Bayes Hilbert spaces are frequently used due to their inherent connection to Bayes's theorem as they allow sampling from potentially intractable posterior densities, a challenging task until now. In the more general context, we prove that pseudo f -divergences are well-defined and introduce a variational estimation framework that can be used in a statistical learning context. By applying this variational estimation framework to f -GANs, we achieve improved FID scores over existing f -GAN architectures and competitive results with the Wasserstein GAN, highlighting its potential for both theoretical research and practical applications in learning theory.