Convergence of Mean-Field Langevin Stochastic Descent-Ascent for Distributional Minimax Optimization
Zhangyi Liu, Feng Liu, Rui Gao, Shuang Li
Abstract
We study convergence properties of the discretetime Mean-Field Langevin Stochastic Descent-Ascent (MFL-SDA) algorithm for solving distributional minimax optimization. These problems arise in various applications, such as zero-sum games, generative adversarial networks and distributionally robust learning. Despite the significance of MFL-SDA in these contexts, the discretetime convergence rate remains underexplored. To address this gap, we establish a last-iterate convergence rate of O( 1 ϵ log 1 ϵ ) for MFL-SDA. This rate is nearly optimal when compared to the complexity lower bound of its Euclidean counterpart. This rate also matches the complexity of mean-field Langevin stochastic gradient descent for distributional minimization and the outer-loop iteration complexity of an existing double-loop algorithm for distributional minimax problems. By leveraging an elementary analysis framework that avoids PDE-based techniques, we overcome previous limitations and achieve a faster convergence rate.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext aca8ed63-275d-45de-9f08-701cb273fd7fBuilds on12
- On Gradient Descent Ascent for Nonconvex-Concave Minimax ProblemsTianyi Lin, Chi Jin, Michael I. JordanICML 2020 · 587 citations
- What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?Chi Jin, Praneeth Netrapalli, Michael I. JordanICML 2020 · 381 citations
- Global Convergence and Variance Reduction for a Class of Nonconvex-Nonconcave Minimax ProblemsJunchi Yang, Negar Kiyavash, Niao HeNeurIPS 2020 · 136 citations
- Optimal Epoch Stochastic Gradient Descent Ascent Methods for Min-Max OptimizationYan Yan, Yi Xu, Qihang Lin, Wei Liu et al.NeurIPS 2020 · 70 citations
- A mean-field analysis of two-player zero-sum gamesCarles Domingo-Enrich, Samy Jelassi, Arthur Mensch, Grant M. Rotskoff et al.NeurIPS 2020 · 56 citations
Related papers
- High-probability complexity bounds for stochastic non-convex minimax optimizationYassine Laguel, Yasa Syed, Necdet Serhat Aybat, Mert GürbüzbalabanNeurIPS 2024 · 2 citations
- Efficient Generalization with Distributionally Robust LearningSoumyadip Ghosh, Mark S. Squillante, Ebisa D. WollegaNeurIPS 2021 · 4 citations
- Symmetric Mean-field Langevin Dynamics for Distributional Minimax ProblemsJuno Kim, Kakei Yamamoto, Kazusato Oko, Zhuoran Yang et al.ICLR 2024 · 14 citations
- Delving into the Convergence of Generalized Smooth Minimax OptimizationWenhan Xian, Ziyi Chen, Heng HuangICML 2024 · 7 citations
- A Communication-efficient Algorithm with Linear Convergence for Federated Minimax LearningZhenyu Sun, Ermin WeiNeurIPS 2022 · 20 citations
