AIRE-Prune: Asymptotic Impulse-Response Energy for State Pruning in State Space Models
Apurba Prasad Padhy, Fernando Camacho, Saibal Mukhopadhyay
摘要
State space models (SSMs) often sacrifice capacity, search space, or stability to offset the memory and compute costs of large state dimensions. We introduce a structured post-training pruning method for SSMs -AIRE-Prune (Asymptotic Impulse-Response Energy for State PRUN(E)ing ) -that reduces each layer's state dimension by directly minimizing long-run output-energy distortion. AIRE-Prune assigns every state a closed-form asymptotic impulse-response energy based score, i.e., the total impulse-response energy it contributes over an infinite horizon (time), and normalizes these scores layer-wise to enable global cross-layer comparison and selection. This extends modal truncation from single systems to deep stacks and aligns pruning with asymptotic response energy rather than worstcase gain. Across diverse sequence benchmarks, AIRE-Prune reveals substantial redundancy in SISO and MIMO SSMs with average pruning of 60.8% , with average accuracy drop of 0.29% without retraining while significantly lowering compute. Code will be released: https://github.com/falcon-arrow/AIRE-Prune .
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper12
- Efficiently Modeling Long Sequences with Structured State SpacesAlbert Gu, Karan Goel, Christopher RéICLR 2022 · 被引用 3,482 次
- Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space LayersAlbert Gu, Isys Johnson, Karan Goel, Khaled Saab 等NeurIPS 2021 · 被引用 1,280 次
- HiPPO: Recurrent Memory with Optimal Polynomial ProjectionsAlbert Gu, Tri Dao, Stefano Ermon, Atri Rudra 等NeurIPS 2020 · 被引用 1,100 次
- Long Range Arena : A Benchmark for Efficient TransformersYi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen 等ICLR 2021 · 被引用 881 次
- On the Parameterization and Initialization of Diagonal State Space ModelsAlbert Gu, Karan Goel, Ankit Gupta, Christopher RéNeurIPS 2022 · 被引用 690 次
相关 Paper
- Layer-Adaptive State Pruning for Deep State Space ModelsMinseon Gwak, Seongrok Moon, Joohwan Ko, PooGyeon ParkNeurIPS 2024 · 被引用 13 次
- Efficient Unstructured Pruning of Mamba State-Space Models for Resource-Constrained EnvironmentsIbne Farabi Shihab, Sanjeda Akter, Anuj SharmaEMNLP 2025 · 被引用 1 次
- The Curious Case of In-Training Compression of State Space ModelsMakram Chahine, Philipp Nazari, Daniela Rus, T. Konstantin RuschICLR 2026 · 被引用 4 次
- Rethinking Token Reduction for State Space ModelsZheng Zhan, Yushu Wu, Zhenglun Kong, Changdi Yang 等EMNLP 2024 · 被引用 4 次
- BMRS: Bayesian Model Reduction for Structured PruningDustin Wright, Christian Igel, Raghavendra SelvanNeurIPS 2024 · 被引用 7 次
