Stability beyond Bounded Differences: Sharp Generalization Bounds under Finite Moments
Qianqian Lei, Soham Bonnerjee, Yuefeng Han, Wei Biao Wu
Abstract
While algorithmic stability is a central tool for understanding generalization of learning algorithms, existing high-probability guarantees typically rely on uniform boundedness or sub-Gaussian/sub-Weibull tail assumptions, which can be overly restrictive for modern settings with heavy-tailed or unbounded losses. We develop a stability-based framework that requires only a finite moment condition. Our first contribution is sharp concentration inequalities for functions of independent random variables under constraints, extending McDiarmid's bounded-differences techniques beyond the classical regime. Leveraging these results, we derive sharp high-probability generalization bounds across a range of learning paradigms, including empirical risk minimization, transductive regression, and meta-learning. These guarantees show that stability suffices for robust generalization even when boundedness fails, substantially weakening the standard assumptions in the stability literature.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext c7c8ce31-c100-4ec5-8ed3-a468bf856742Builds on13
- Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient DescentYunwen Lei, Yiming YingICML 2020 · 165 citations
- Convergence of Meta-Learning with Task-Specific Adaptation over Partial ParametersKaiyi Ji, Jason D. Lee, Yingbin Liang, H. Vincent PoorNeurIPS 2020 · 97 citations
- Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen TasksAlireza Fallah, Aryan Mokhtari, Asuman E. OzdaglarNeurIPS 2021 · 63 citations
- Few-Shot Learning via Learning the Representation, ProvablySimon Shaolei Du, Wei Hu, Sham M. Kakade, Jason D. Lee et al.ICLR 2021 · 56 citations
- A Closer Look at the Training Strategy for Modern Meta-LearningJiaxin Chen, Xiao-Ming Wu, Yanke Li, Qimai Li et al.NeurIPS 2020 · 48 citations
Related papers
- Algorithmic Stability Unleashed: Generalization Bounds with Unbounded LossesShaojie Li, Bowei Zhu, Yong LiuICML 2024 · 3 citations
- Concentration Inequalities for General Functions of Heavy-Tailed Random VariablesShaojie Li, Yong LiuICML 2024 · 1 citation
- L2-Uniform Stability of Randomized Learning Algorithms: Sharper Generalization Bounds and Confidence BoostingXiaotong Yuan, Ping LiNeurIPS 2023 · 2 citations
- Toward Better PAC-Bayes Bounds for Uniformly Stable AlgorithmsSijia Zhou, Yunwen Lei, Ata KabánNeurIPS 2023 · 4 citations
- Stability and Generalization of Nonconvex Optimization with Heavy-Tailed NoiseHongxu Chen, Ke Wei, Xiaoming Yuan, Luo LuoICML 2026
