Stability beyond Bounded Differences: Sharp Generalization Bounds under Finite Moments
Qianqian Lei, Soham Bonnerjee, Yuefeng Han, Wei Biao Wu
摘要
While algorithmic stability is a central tool for understanding generalization of learning algorithms, existing high-probability guarantees typically rely on uniform boundedness or sub-Gaussian/sub-Weibull tail assumptions, which can be overly restrictive for modern settings with heavy-tailed or unbounded losses. We develop a stability-based framework that requires only a finite moment condition. Our first contribution is sharp concentration inequalities for functions of independent random variables under constraints, extending McDiarmid's bounded-differences techniques beyond the classical regime. Leveraging these results, we derive sharp high-probability generalization bounds across a range of learning paradigms, including empirical risk minimization, transductive regression, and meta-learning. These guarantees show that stability suffices for robust generalization even when boundedness fails, substantially weakening the standard assumptions in the stability literature.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper13
- Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient DescentYunwen Lei, Yiming YingICML 2020 · 被引用 165 次
- Convergence of Meta-Learning with Task-Specific Adaptation over Partial ParametersKaiyi Ji, Jason D. Lee, Yingbin Liang, H. Vincent PoorNeurIPS 2020 · 被引用 97 次
- Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen TasksAlireza Fallah, Aryan Mokhtari, Asuman E. OzdaglarNeurIPS 2021 · 被引用 63 次
- Few-Shot Learning via Learning the Representation, ProvablySimon Shaolei Du, Wei Hu, Sham M. Kakade, Jason D. Lee 等ICLR 2021 · 被引用 56 次
- A Closer Look at the Training Strategy for Modern Meta-LearningJiaxin Chen, Xiao-Ming Wu, Yanke Li, Qimai Li 等NeurIPS 2020 · 被引用 48 次
相关 Paper
- Algorithmic Stability Unleashed: Generalization Bounds with Unbounded LossesShaojie Li, Bowei Zhu, Yong LiuICML 2024 · 被引用 3 次
- Concentration Inequalities for General Functions of Heavy-Tailed Random VariablesShaojie Li, Yong LiuICML 2024 · 被引用 1 次
- L2-Uniform Stability of Randomized Learning Algorithms: Sharper Generalization Bounds and Confidence BoostingXiaotong Yuan, Ping LiNeurIPS 2023 · 被引用 2 次
- Toward Better PAC-Bayes Bounds for Uniformly Stable AlgorithmsSijia Zhou, Yunwen Lei, Ata KabánNeurIPS 2023 · 被引用 4 次
- Stability and Generalization of Nonconvex Optimization with Heavy-Tailed NoiseHongxu Chen, Ke Wei, Xiaoming Yuan, Luo LuoICML 2026
