Efficient Sliced Wasserstein Distance Computation via Adaptive Bayesian Optimization
Manish Acharya, David Hyde
Abstract
The sliced Wasserstein distance (SW) reduces optimal transport on to a sum of one-dimensional projections, and thanks to this efficiency, it is widely used in geometry, generative modeling, and registration tasks. Recent work shows that quasi-Monte Carlo constructions for computing SW (QSW) yield direction sets with excellent approximation error. This paper presents an alternate, novel approach: learning directions with Bayesian optimization (BO), particularly in settings where SW appears inside an optimization loop (e.g., gradient flows). We introduce a family of drop-in selectors for projection directions: BOSW, a one-shot BO scheme on the unit sphere; RBOSW, a periodic-refresh variant; ABOSW, an adaptive hybrid that seeds from competitive QSW sets and performs a few lightweight BO refinements; and ARBOSW, a restarted hybrid that periodically relearns directions during optimization. Our BO approaches can be composed with QSW and its variants (demonstrated by ABOSW/ARBOSW) and require no changes to downstream losses or gradients. We provide numerical experiments where our methods achieve state-of-the-art performance, and on the experimental suite of the original QSW paper, we find that ABOSW and ARBOSW can achieve convergence comparable to the best QSW variants with modest runtime overhead. We release code with fixed seeds and configurations to support faithful replication (see supplementary material).
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 7d7f6522-f032-4d46-9bb2-d75f76dd6036Builds on10
- BoTorch: A Framework for Efficient Monte-Carlo Bayesian OptimizationMaximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton et al.NeurIPS 2020 · 686 citations
- Unexpected Improvements to Expected Improvement for Bayesian OptimizationSebastian Ament, Samuel Daulton, David Eriksson, Maximilian Balandat et al.NeurIPS 2023 · 280 citations
- Matérn Gaussian Processes on Riemannian ManifoldsViacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, Marc Peter DeisenrothNeurIPS 2020 · 151 citations
- Statistical and Topological Properties of Sliced Probability DivergencesKimia Nadjahi, Alain Durmus, Lénaïc Chizat, Soheil Kolouri et al.NeurIPS 2020 · 115 citations
- Vanilla Bayesian Optimization Performs Great in High DimensionsCarl Hvarfner, Erik Orm Hellsten, Luigi NardiICML 2024 · 88 citations
Related papers
- Markovian Sliced Wasserstein Distances: Beyond Independent ProjectionsKhai Nguyen, Tongzheng Ren, Nhat HoNeurIPS 2023 · 13 citations
- Quasi-Monte Carlo for 3D Sliced WassersteinKhai Nguyen, Nicola Bariletto, Nhat HoICLR 2024 · 25 citations
- Distributional Sliced-Wasserstein and Applications to Generative ModelingKhai Nguyen, Nhat Ho, Tung Pham, Hung BuiICLR 2021 · 111 citations
- Fast Approximation of the Sliced-Wasserstein Distance Using Concentration of Random ProjectionsKimia Nadjahi, Alain Durmus, Pierre E. Jacob, Roland Badeau et al.NeurIPS 2021 · 54 citations
- Energy-Based Sliced Wasserstein DistanceKhai Nguyen, Nhat HoNeurIPS 2023 · 51 citations
