Guarantees of a Preconditioned Subgradient Algorithm for Overparameterized Asymmetric Low-rank Matrix Recovery
Paris Giampouras, HanQin Cai, René Vidal
Abstract
In this paper, we focus on a matrix factorizationbased approach to recover low-rank asymmetric matrices from corrupted measurements. We propose an Overparameterized Preconditioned Subgradient Algorithm (OPSA) and provide, for the first time in the literature, linear convergence rates independent of the rank of the sought asymmetric matrix in the presence of gross corruptions. Our work goes beyond existing results in preconditioned-type approaches addressing their current limitation, i.e., the lack of convergence guarantees in the case of asymmetric matrices of unknown rank. By applying our approach to (robust) matrix sensing, we highlight its merits when the measurement operator satisfies a mixednorm restricted isometry property. Lastly, we present extensive numerical experiments that validate our theoretical results and demonstrate the effectiveness of our approach for different levels of overparameterization and outlier corruptions.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 69f2ccc2-2f76-4ffb-898a-7ef824177d85Cited by top-tier papers2
- The Power of Preconditioning in Overparameterized Low-Rank Matrix SensingXingyu Xu, Yandi Shen, Yuejie Chi, Cong MaICML 2023 · 51 citations
- LoRA-One: One-Step Full Gradient Could Suffice for Fine-Tuning Large Language Models, Provably and EfficientlyYuanhe Zhang, Fanghui Liu, Yudong ChenICML 2025
Builds on8
- Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstructionDominik Stöger, Mahdi SoltanolkotabiNeurIPS 2021 · 101 citations
- Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier DetectionHanQin Cai, Jialin Liu, Wotao YinNeurIPS 2021 · 69 citations
- The Power of Preconditioning in Overparameterized Low-Rank Matrix SensingXingyu Xu, Yandi Shen, Yuejie Chi, Cong MaICML 2023 · 51 citations
- Riemannian Preconditioned LoRA for Fine-Tuning Foundation ModelsFangzhao Zhang, Mert PilanciICML 2024 · 43 citations
- Rank Overspecified Robust Matrix Recovery: Subgradient Method and Exact RecoveryLijun Ding, Liwei Jiang, Yudong Chen, Qing Qu et al.NeurIPS 2021 · 30 citations
Related papers
- How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and InitializationNuoya Xiong, Lijun Ding, Simon Shaolei DuICLR 2024 · 22 citations
- Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix FactorizationTian Ye, Simon S. DuNeurIPS 2021 · 61 citations
- Local and Global Linear Convergence of General Low-Rank Matrix Recovery ProblemsYingjie Bi, Haixiang Zhang, Javad LavaeiAAAI 2022 · 21 citations
- Preconditioned Gradient Descent for Over-Parameterized Nonconvex Matrix FactorizationJialun Zhang, Salar Fattahi, Richard Y. ZhangNeurIPS 2021 · 47 citations
- RGNMR: A Gauss-Newton method for robust matrix completion with theoretical guaranteesEilon Vaknin Laufer, Boaz NadlerNeurIPS 2025 · 1 citation
