AIRE-Prune: Asymptotic Impulse-Response Energy for State Pruning in State Space Models
Apurba Prasad Padhy, Fernando Camacho, Saibal Mukhopadhyay
Abstract
State space models (SSMs) often sacrifice capacity, search space, or stability to offset the memory and compute costs of large state dimensions. We introduce a structured post-training pruning method for SSMs -AIRE-Prune (Asymptotic Impulse-Response Energy for State PRUN(E)ing ) -that reduces each layer's state dimension by directly minimizing long-run output-energy distortion. AIRE-Prune assigns every state a closed-form asymptotic impulse-response energy based score, i.e., the total impulse-response energy it contributes over an infinite horizon (time), and normalizes these scores layer-wise to enable global cross-layer comparison and selection. This extends modal truncation from single systems to deep stacks and aligns pruning with asymptotic response energy rather than worstcase gain. Across diverse sequence benchmarks, AIRE-Prune reveals substantial redundancy in SISO and MIMO SSMs with average pruning of 60.8% , with average accuracy drop of 0.29% without retraining while significantly lowering compute. Code will be released: https://github.com/falcon-arrow/AIRE-Prune .
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 8ac6b1e7-32b5-4c0e-9d41-ac1b01e9a0e4Builds on12
- Efficiently Modeling Long Sequences with Structured State SpacesAlbert Gu, Karan Goel, Christopher RéICLR 2022 · 3,482 citations
- Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space LayersAlbert Gu, Isys Johnson, Karan Goel, Khaled Saab et al.NeurIPS 2021 · 1,280 citations
- HiPPO: Recurrent Memory with Optimal Polynomial ProjectionsAlbert Gu, Tri Dao, Stefano Ermon, Atri Rudra et al.NeurIPS 2020 · 1,100 citations
- Long Range Arena : A Benchmark for Efficient TransformersYi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen et al.ICLR 2021 · 881 citations
- On the Parameterization and Initialization of Diagonal State Space ModelsAlbert Gu, Karan Goel, Ankit Gupta, Christopher RéNeurIPS 2022 · 690 citations
Related papers
- Layer-Adaptive State Pruning for Deep State Space ModelsMinseon Gwak, Seongrok Moon, Joohwan Ko, PooGyeon ParkNeurIPS 2024 · 13 citations
- Efficient Unstructured Pruning of Mamba State-Space Models for Resource-Constrained EnvironmentsIbne Farabi Shihab, Sanjeda Akter, Anuj SharmaEMNLP 2025 · 1 citation
- The Curious Case of In-Training Compression of State Space ModelsMakram Chahine, Philipp Nazari, Daniela Rus, T. Konstantin RuschICLR 2026 · 4 citations
- Rethinking Token Reduction for State Space ModelsZheng Zhan, Yushu Wu, Zhenglun Kong, Changdi Yang et al.EMNLP 2024 · 4 citations
- BMRS: Bayesian Model Reduction for Structured PruningDustin Wright, Christian Igel, Raghavendra SelvanNeurIPS 2024 · 7 citations
