Convergence of Mean-Field Langevin Stochastic Descent-Ascent for Distributional Minimax Optimization
Zhangyi Liu, Feng Liu, Rui Gao, Shuang Li
摘要
We study convergence properties of the discretetime Mean-Field Langevin Stochastic Descent-Ascent (MFL-SDA) algorithm for solving distributional minimax optimization. These problems arise in various applications, such as zero-sum games, generative adversarial networks and distributionally robust learning. Despite the significance of MFL-SDA in these contexts, the discretetime convergence rate remains underexplored. To address this gap, we establish a last-iterate convergence rate of O( 1 ϵ log 1 ϵ ) for MFL-SDA. This rate is nearly optimal when compared to the complexity lower bound of its Euclidean counterpart. This rate also matches the complexity of mean-field Langevin stochastic gradient descent for distributional minimization and the outer-loop iteration complexity of an existing double-loop algorithm for distributional minimax problems. By leveraging an elementary analysis framework that avoids PDE-based techniques, we overcome previous limitations and achieve a faster convergence rate.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper12
- On Gradient Descent Ascent for Nonconvex-Concave Minimax ProblemsTianyi Lin, Chi Jin, Michael I. JordanICML 2020 · 被引用 587 次
- What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?Chi Jin, Praneeth Netrapalli, Michael I. JordanICML 2020 · 被引用 381 次
- Global Convergence and Variance Reduction for a Class of Nonconvex-Nonconcave Minimax ProblemsJunchi Yang, Negar Kiyavash, Niao HeNeurIPS 2020 · 被引用 136 次
- Optimal Epoch Stochastic Gradient Descent Ascent Methods for Min-Max OptimizationYan Yan, Yi Xu, Qihang Lin, Wei Liu 等NeurIPS 2020 · 被引用 70 次
- A mean-field analysis of two-player zero-sum gamesCarles Domingo-Enrich, Samy Jelassi, Arthur Mensch, Grant M. Rotskoff 等NeurIPS 2020 · 被引用 56 次
相关 Paper
- High-probability complexity bounds for stochastic non-convex minimax optimizationYassine Laguel, Yasa Syed, Necdet Serhat Aybat, Mert GürbüzbalabanNeurIPS 2024 · 被引用 2 次
- Efficient Generalization with Distributionally Robust LearningSoumyadip Ghosh, Mark S. Squillante, Ebisa D. WollegaNeurIPS 2021 · 被引用 4 次
- Symmetric Mean-field Langevin Dynamics for Distributional Minimax ProblemsJuno Kim, Kakei Yamamoto, Kazusato Oko, Zhuoran Yang 等ICLR 2024 · 被引用 14 次
- Delving into the Convergence of Generalized Smooth Minimax OptimizationWenhan Xian, Ziyi Chen, Heng HuangICML 2024 · 被引用 7 次
- A Communication-efficient Algorithm with Linear Convergence for Federated Minimax LearningZhenyu Sun, Ermin WeiNeurIPS 2022 · 被引用 20 次
