Lune

ACL Findings2026

CoopQ: Cooperative Game Inspired Layerwise Mixed Precision Quantization for LLMs

Junchen Zhao, Ali Derakhshan, Jayden Kana Hyman, Junhao Dong, Sangeetha Abdu Jyothi, Ian Harris

Abstract

Large Language Models (LLMs) promise impressive capabilities, yet their multi-billionparameter scale makes on-device or lowresource deployment prohibitive. Mixedprecision quantization offers a compelling solution, but existing methods struggle when the average precision drops below four bits, as they rely on isolated, layer-specific metrics that overlook critical inter-layer interactions affecting overall performance. To address these limitations, we first frame the mixed-precision quantization problem as a cooperative game among layers and introduce Shapley-based Progressive Quantization Estimation (SPQE) to efficiently obtain accurate Shapley estimates of layer sensitivities and inter-layer interactions. Leveraging the SPQE estimates, we propose Cooperative Game Inspired Mixed-Precision Quantization (CoopQ) which translates these Shapley estimates into a binary quadratic optimization formulation, assigning either 2 or 4-bit precision to layers under strict memory constraints. Comprehensive experiments conducted on Llama-3, Gemma-2, and Qwen-3 models across three independent PTQ backends (Quanto, HQQ, GPTQ) demonstrate CoopQ's scalability and consistently superior performance compared to methods relying solely on isolated metrics. Across average precisions spanning 4 bit down to 2 bit, CoopQ cuts Perplexity by 20 -80 % relative to the best baseline, with the margin growing as the bit-width tightens.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression cites SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
  • SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression cites Language Models are Few-Shot Learners
  • ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers cites Language Models are Few-Shot Learners
  • SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on10

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language ModelsGuangxuan Xiao, Ji Lin, Mickaël Seznec, Hao Wu et al.ICML 2023 · 1,493 citations
  3. Compressive Transformers for Long-Range Sequence ModellingJack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Chloe Hillier et al.ICLR 2020 · 833 citations
  4. ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale TransformersZhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu et al.NeurIPS 2022 · 816 citations
  5. SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight CompressionTim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev et al.ICLR 2024 · 392 citations

Related papers

  1. SKIM: Any-bit Quantization Pushing The Limits of Post-Training QuantizationRunsheng Bai, Bo Liu, Qiang LiuICML 2025
  2. GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMsJianing Deng, Song Wang, Dongwei Wang, Zijie Liu et al.ICML 2026 · 3 citations
  3. SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language ModelsWei Huang, Haotong Qin, Yangdong Liu, Yawei Li et al.ICML 2025
  4. Exploring Layer-wise Information Effectiveness for Post-Training Quantization in Small Language ModelsHe Xiao, Qingyao Yang, Dirui Xie, Wendong Xu et al.ACL Findings 2026
  5. GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language ModelsPengxiang Zhao, Xiaoming YuanICML 2025
Dusk over the sea between two cliffs drawn in fine vertical lines