Lune

NeurIPS2025Top-tier venue

Generalization Error Analysis for Selective State-Space Models Through the Lens of Attention

Arya Honarpisheh, Mustafa Bozdag, Octavia I. Camps, Mario Sznaier

2025Year
6Citations
2Top-tier citations

Abstract

State-space models (SSMs) have recently emerged as a compelling alternative to Transformers for sequence modeling tasks. This paper presents a theoretical generalization analysis of selective SSMs, the core architectural component behind the Mamba model. We derive a novel covering number-based generalization bound for selective SSMs, building upon recent theoretical advances in the analysis of Transformer models. Using this result, we analyze how the spectral abscissa of the continuous-time state matrix influences the model's stability during training and its ability to generalize across sequence lengths. We empirically validate our findings on a synthetic majority task, the IMDb sentiment classification benchmark, and the ListOps task, demonstrating how our theoretical insights translate into practical model behavior.

Ask about this paper

Your agent reads all of it.

Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.

Questions to start from

Your agent calls

Luneget_paper_fulltext

Ask in Lune

Free to start. No credit card required.

lune papers fulltext 0a3899b8-ba61-4ea5-b3cf-c41b387b0fcf

Cited by top-tier papers2

Ask how each one uses it

Builds on31

Related papers

Dusk over the sea between two cliffs drawn in fine vertical lines