Multiobjective distribution matching
Xiaoyuan Zhang, Peijie Li, Yingying Yu, Yichi Zhang, Han Zhao, Qingfu Zhang
Abstract
Distribution matching is a core concept in machine learning, with applications in generative models, domain adaptation, and algorithmic fairness. A closely related but less explored challenge is generating a distribution that aligns with multiple underlying distributions, often with conflicting objectives, known as a Pareto optimal distribution. In this paper, we develop a general theory based on information geometry to construct the Pareto set and front for the entire exponential family under KL and inverse KL divergences. This formulation allows explicit derivation of the Pareto set and front for multivariate normal distributions, enabling applications like multiobjective variational autoencoders (MOVAEs) to generate interpolated image distributions. Experimental results on real-world images demonstrate that both algorithms can generate high-quality interpolated images across multiple distributions.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on17
- Multi-Objective Molecule Generation using Interpretable SubstructuresWengong Jin, Regina Barzilay, Tommi S. JaakkolaICML 2020 · 238 citations
- Domain Adaptation with Conditional Distribution Matching and Generalized Label ShiftRemi Tachet des Combes, Han Zhao, Yu-Xiang Wang, Geoffrey J. GordonNeurIPS 2020 · 231 citations
- Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain GeneralizationYabin Zhang, Minghan Li, Ruihuang Li, Kui Jia et al.CVPR 2022 · 217 citations
- Learning the Pareto Front with HypernetworksAviv Navon, Aviv Shamsian, Ethan Fetaya, Gal ChechikICLR 2021 · 189 citations
- Multi-Task Learning with User Preferences: Gradient Descent with Controlled Ascent in Pareto OptimizationDebabrata Mahapatra, Vaibhav RajanICML 2020 · 182 citations
Related papers
- Fair Domain Generalization: An Information-Theoretic ViewTangzheng Lian, Guanyu Hu, Dimitrios Kollias, Xinyu Yang et al.AAAI 2026 · 1 citation
- Multimarginal Generative Modeling with Stochastic InterpolantsMichael S. Albergo, Nicholas Matthew Boffi, Michael Lindsey, Eric Vanden-EijndenICLR 2024 · 19 citations
- Pareto Variational AutoencoderMincheol Cho, Yedarm Seong, Joong-Ho WonICLR 2026
- Pareto-frontier Entropy Search with Variational Lower Bound MaximizationMasanori Ishikura, Masayuki KarasuyamaICML 2025
- A General Approach to Fairness with Optimal TransportSilvia Chiappa, Ray Jiang, Tom Stepleton, Aldo Pacchiano et al.AAAI 2020 · 94 citations
