Optimal Spectral Transitions in High-Dimensional Multi-Index Models
Leonardo Defilippis, Yatin Dandi, Pierre Mergny, Florent Krzakala, Bruno Loureiro
摘要
We consider the problem of how many samples from a Gaussian multi-index model are required to weakly reconstruct the relevant index subspace. Despite its increasing popularity as a testbed for investigating the computational complexity of neural networks, results beyond the single-index setting remain elusive. In this work, we introduce spectral algorithms based on the linearization of a message passing scheme tailored to this problem. Our main contribution is to show that the proposed methods achieve the optimal reconstruction threshold. Leveraging a high-dimensional characterization of the algorithms, we show that above the critical threshold the leading eigenvector correlates with the relevant index subspace, a phenomenon reminiscent of the Baik-Ben Arous-Peche (BBP) transition in spiked models arising in random matrix theory. Supported by numerical experiments and a rigorous theoretical framework, our work bridges critical gaps in the computational limits of weak learnability in multi-index model.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper7
- Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural NetworksDaniel Kunin, Giovanni Luca Marchetti, Feng Chen, Dhruva Karkada 等NeurIPS 2025 · 被引用 15 次
- Learning single index models via harmonic decompositionNirmit Joshi, Hugo Koubbi, Theodor Misiakiewicz, Nati SrebroNeurIPS 2025 · 被引用 8 次
- Algorithms and SQ Lower Bounds for Robustly Learning Real-valued Multi-Index ModelsIlias Diakonikolas, Giannis Iakovidis, Daniel Kane, Lisheng RenNeurIPS 2025 · 被引用 8 次
- Neural Networks Learn Generic Multi-Index Models Near Information-Theoretic LimitBohan Zhang, Zihao Wang, Hengyu Fu, Jason D. LeeICLR 2026 · 被引用 3 次
- Full-Batch Gradient Descent Outperforms One-Pass SGD: Sample Complexity Separation in Single-Index LearningFilip Kovačević, Hong Chang Ji, Denny Wu, Mahdi Soltanolkotabi 等ICML 2026 · 被引用 2 次
它引用的顶会 Paper8
- Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networksRodrigo Veiga, Ludovic Stephan, Bruno Loureiro, Florent Krzakala 等NeurIPS 2022 · 被引用 59 次
- Neural network learns low-dimensional polynomials with SGD near the information-theoretic limitJason D. Lee, Kazusato Oko, Taiji Suzuki, Denny WuNeurIPS 2024 · 被引用 49 次
- The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap ExponentsYatin Dandi, Emanuele Troiani, Luca Arnaboldi, Luca Pesce 等ICML 2024 · 被引用 41 次
- Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message PassingRamji Venkataramanan, Kevin Kögler, Marco MondelliICML 2022 · 被引用 36 次
- Optimal Algorithms for the Inhomogeneous Spiked Wigner ModelAleksandr Pak, Justin Ko, Florent KrzakalaNeurIPS 2023 · 被引用 17 次
相关 Paper
- Overparametrization bends the landscape: BBP transitions at initialization in simple Neural NetworksBrandon Livio Annesi, Dario Bocchi, Chiara CammarotaICLR 2026 · 被引用 3 次
- Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High DimensionsYizhou Xu, Florent Krzakala, Lenka ZdeborováNeurIPS 2025 · 被引用 1 次
- Detection of Signal in the Spiked Rectangular ModelsJi Hyung Jung, Hye Won Chung, Ji Oon LeeICML 2021 · 被引用 11 次
- Subspace clustering in high-dimensions: Phase transitions & Statistical-to-Computational gapLuca Pesce, Bruno Loureiro, Florent Krzakala, Lenka ZdeborováNeurIPS 2022 · 被引用 4 次
- Fluctuations of the largest eigenvalues of transformed spiked Wigner matricesAro Lee, Ji Oon LeeICML 2025
