Energy-guided Entropic Neural Optimal Transport

Energy-based models (EBMs) are known in the Machine Learning community for decades. Since the seminal works devoted to EBMs dating back to the noughties, there have been a lot of efficient methods which solve the generative modelling problem by means of energy potentials (unnormalized likelihood functions). In contrast, the realm of Optimal Transport (OT) and, in particular, neural OT solvers is much less explored and limited by few recent works (excluding WGAN-based approaches which utilize OT as a loss function and do not model OT maps themselves). In our work, we bridge the gap between EBMs and Entropy-regularized OT. We present a novel methodology which allows utilizing the recent developments and technical improvements of the former in order to enrich the latter. From the theoretical perspective, we prove generalization bounds for our technique. In practice, we validate its applicability in toy 2D and image domains. To showcase the scalability, we empower our method with a pre-trained StyleGAN and apply it to high-res AFHQ 512 × 512 unpaired I2I translation. For simplicity, we choose simple short-and long-run EBMs as a backbone of our Energy-guided Entropic OT approach, leaving the application of more sophisticated EBMs for future research. Our code is available at: https: //github.com/PetrMokrov/Energy-guided-Entropic-OT Figure 1: AFHQ 512 × 512 Cat→Dog unpaired translation by our Energy-guided EOT solver applied in the latent space of StyleGAN2-ADA. Our approach does not need data2latent encoding. Left: source samples; right: translated samples.