Lune

ACL Findings2026

Activation Steering for Chain-of-Thought Compression

Seyedarmin Azizi, Erfan Baghaei Potraghloo, Souvik Kundu, Massoud Pedram

Abstract

Large language models (LLMs) demonstrate strong performance on multi-step reasoning tasks by producing intermediate explanations, commonly referred to as chains of thought (CoTs). However, the generated rationales are typically verbose, consuming many additional tokens, and thus degrading throughput and increasing inference energy consumption. Interestingly, we find that verbose and concise CoTs correspond to distinct regions in the model's intermediate activation space, suggesting that verbosity is a steerable latent attribute. Building on this observation, we develop an inference-time method to automatically steer the model response towards concise reasoning traces without updating model parameters. Our method, dubbed ASC (Activation-Steered Compression), generates concise CoTs by directly adjusting internal representations via activation steering. A key component of ASC is Contrastive Energy-Based Steering (CES), a principled procedure to learn a single steering vector from a small set of verbose-concise CoT pairs by optimizing a lengthnormalized contrastive energy objective. To further ensure reliable steering and preserve general utility, CES enforces a differentiable KL trust region during steering vector optimization, explicitly constraining the distribution shift within a specified budget. With only 100 pairs of verbose-concise examples, ASC reduces the generated token length by as much as 69.4% across five reasoning benchmarks (MATH500, GSM8K, LiveCodeBench, GSM8K-Hard, and AQuA-RAT) while maintaining accuracy across models with 1.5B, 7B, 8B, and 32B parameters. On MATH500, ASC achieves an end-to-end inference speed-up of 2.7× on an 8B model.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Merlin's Whisper: Enabling Efficient Reasoning in Large Language Models via Black-box Persuasive Prompting cites Dynamic Early Exit in Reasoning Models
  • Merlin's Whisper: Enabling Efficient Reasoning in Large Language Models via Black-box Persuasive Prompting cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites PAL: Program-aided Language Models
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • No More, No Less: Least-Privilege Language Models cites Locating and Editing Factual Associations in GPT
  • PAL: Program-aided Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Dynamic Early Exit in Reasoning Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Merlin's Whisper: Enabling Efficient Reasoning in Large Language Models via Black-box Persuasive PromptingHeming Xia, Cunxiao Du, Rui Li, Chak Tou Leong et al.ACL 2026
  2. WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient ReasoningGagan Mundada, Zihan Huang, Rohan Surana, Sheldon Yu et al.ICML 2026
  3. No More, No Less: Least-Privilege Language ModelsPaulius Rauba, Dominykas Seputis, Patrikas Vanagas, Mihaela van der SchaarICML 2026

Builds on11

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
  3. Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
  4. PAL: Program-aided Language ModelsLuyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon et al.ICML 2023 · 700 citations
  5. Dynamic Early Exit in Reasoning ModelsChenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu et al.ICLR 2026 · 250 citations

Related papers

  1. Self-Training Elicits Concise Reasoning in Large Language ModelsTergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang et al.ACL Findings 2025 · 3 citations
  2. ConPress: Learning Efficient Reasoning from Multi-Question Contextual PressureJie Deng, Shining Liang, Jun Li, Hongzhi Li et al.ICML 2026 · 3 citations
  3. System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic ShortcutsXiaoqiang Wang, Suyuchen Wang, Yun Zhu, Bang LiuNeurIPS 2025 · 26 citations
  4. SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning ModelsKerui Huang, Shuhan Liu, Xing Hu, Tongtong Xu et al.ISSTA 2026
  5. C3oT: Generating Shorter Chain-of-Thought Without Compromising EffectivenessYu Kang, Xianghui Sun, Liangyu Chen, Wei ZouAAAI 2025 · 162 citations
Dusk over the sea between two cliffs drawn in fine vertical lines