Lune

ACL Findings2025

Q-Mamba: Towards more efficient Mamba models via post-training quantization

Tianqi Chen, Yuanteng Chen, Peisong Wang, Weixiang Xu, Zeyu Zhu, Jian Cheng

Abstract

State Space Models (SSMs), such as Mamba, have recently demonstrated potential in language understanding tasks, positioning them as competitors to transformer architectures. However, our investigations reveal that the Mamba architecture still has room for further optimization-not only in linear projections but also in state caches, which contribute significantly to memory consumption, particularly after quantizing the former into low bits. After a theoretical analysis of the causes of outliers in states, we propose Decoupled Scale Quantization (DSQ), which mitigates outliers in both the state and channel dimensions by applying separate quantization scales. To preserve the selective ability of quantized Mamba, we introduce Efficient Selectivity Reconstruction (ESR), a novel quantization simulation scheme in block-wise reconstruction that enables fast parallel scan algorithms with the non-linear quantization function. We demonstrate the effectiveness of Q-Mamba across various quantization settings, model sizes, and both generation and zero-shot tasks. In particular, for Mamba2-2.7B with W8A8H4 (8-bit weights and activations, 4-bit state caches) quantization, Q-Mamba achieves a 50% reduction in memory consumption with only a 2.13% average accuracy degradation on zero-shot tasks.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model Serving cites PIQA: Reasoning about Physical Commonsense in Natural Language
  • Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model Serving cites Efficiently Modeling Long Sequences with Structured State Spaces
  • Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model Serving cites HiPPO: Recurrent Memory with Optimal Polynomial Projections
  • Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model Serving cites WinoGrande: An Adversarial Winograd Schema Challenge at Scale
  • Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model Serving cites Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
  • PIQA: Reasoning about Physical Commonsense in Natural Language cites WinoGrande: An Adversarial Winograd Schema Challenge at Scale
  • Efficiently Modeling Long Sequences with Structured State Spaces cites HiPPO: Recurrent Memory with Optimal Polynomial Projections
  • Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality cites PIQA: Reasoning about Physical Commonsense in Natural Language
  • Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality cites Efficiently Modeling Long Sequences with Structured State Spaces
  • Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality cites HiPPO: Recurrent Memory with Optimal Polynomial Projections
  • Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality cites WinoGrande: An Adversarial Winograd Schema Challenge at Scale
  • DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization cites WinoGrande: An Adversarial Winograd Schema Challenge at Scale
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. DartQuant: Efficient Rotational Distribution Calibration for LLM QuantizationYuantian Shao, Yuanteng Chen, Peisong Wang, Jianlin Yu et al.NeurIPS 2025 · 20 citations
  2. Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model ServingWonung Kim, Yubin Lee, Yoonsung Kim, Jinwoo Hwang et al.MICRO 2025 · 6 citations

Builds on19

  1. Efficiently Modeling Long Sequences with Structured State SpacesAlbert Gu, Karan Goel, Christopher RéICLR 2022 · 3,482 citations
  2. WinoGrande: An Adversarial Winograd Schema Challenge at ScaleKeisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin ChoiAAAI 2020 · 3,037 citations
  3. PIQA: Reasoning about Physical Commonsense in Natural LanguageYonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao et al.AAAI 2020 · 2,916 citations
  4. Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space DualityTri Dao, Albert GuICML 2024 · 1,407 citations
  5. HiPPO: Recurrent Memory with Optimal Polynomial ProjectionsAlbert Gu, Tri Dao, Stefano Ermon, Atri Rudra et al.NeurIPS 2020 · 1,100 citations

Related papers

  1. Quamba: A Post-Training Quantization Recipe for Selective State Space ModelsHung-Yueh Chiang, Chi-Chih Chang, Natalia Frumkin, Kai-Chiang Wu et al.ICLR 2025
  2. Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space ModelsHung-Yueh Chiang, Chi-Chih Chang, Natalia Frumkin, Kai-Chiang Wu et al.ICML 2025
  3. Understanding Input Selectivity in Mamba: Impact on Approximation Power, Memorization, and Associative Recall CapacityNingyuan Teresa Huang, Miguel Sarabia, Abhinav Moudgil, Pau Rodríguez et al.ICML 2025
  4. VMamba: Visual State Space ModelYue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu et al.NeurIPS 2024 · 3,199 citations
  5. Mamba-3: Improved Sequence Modeling using State Space PrinciplesAakash Sunil Lahoti, Kevin Y. Li, Berlin Chen, Caitlin Wang et al.ICLR 2026 · 96 citations
Dusk over the sea between two cliffs drawn in fine vertical lines