ACL Findings2026
Valid Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought
DaeYeop Lee, Hwanjo Yu
Abstract
Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Large Language Models (LLMs), yet it often incurs substantial computational costs due to "over-reasoning"-the generation of redundant, verbose, or irrelevant steps. While existing reasoning step evaluators effectively detect logical fallacies and factual errors, our analysis reveals a critical blind spot: they fail to penalize "valid but inefficient" reasoning steps that inflate token usage without contributing to the solution. To systematically diagnose this limitation, we introduce RIV-GSM8K, a diagnostic benchmark injected with five distinct types of inefficiencies, including circular reasoning and excessive decomposition. Diagnostic experiments reveal that state-of-the-art evaluators struggle to distinguish these inefficiencies from necessary reasoning. To address this gap, we propose CAID (Context-Aware Information Density), a training-free metric grounded in information theory that identifies low-utility steps. To validate the metric's practical utility, we apply it within PACE, a post-hoc compression strategy. Additional control experiments show that the gains of PACE are not explained by trivial pruning: compared with random step removal and PRM-based compression baselines, it preserves accuracy at substantially higher compression rates. Empirical results on GSM8K, StrategyQA, and ARC-Challenge demonstrate that PACE reduces token consumption by 31-53% while maintaining accuracy, confirming that CAID successfully distills informational "froth" from reasoning chains without compromising deductive validity.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Let's Verify Step by Step cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Let's Verify Step by Step cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds on13
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought PromptingMiles Turpin, Julian Michael, Ethan Perez, Samuel R. BowmanNeurIPS 2023 · 1,792 citations
- H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language ModelsZhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen et al.NeurIPS 2023 · 1,003 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
Related papers
- Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step EntropyZeju Li, Jianyuan Zhong, Ziyang Zheng, Xiangyu Wen et al.ICLR 2026 · 35 citations
- Thinking Economically: A Hierarchical Framework for Adaptive-Complexity Reasoning in LLMsYubo Gao, Haotian Wu, Hong Chen, Junquan Huang et al.ACL Findings 2026
- Self-Training Elicits Concise Reasoning in Large Language ModelsTergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang et al.ACL Findings 2025 · 3 citations
- Causal Sufficiency and Necessity Improves Chain-of-Thought ReasoningXiangning Yu, Zhuohan Wang, Linyi Yang, Haoxuan Li et al.NeurIPS 2025 · 19 citations
- Can Reasoning Path still be Effective as Input? Bridging Post-Reasoning to Chain-of-Thought CompressionChengzhengxu Li, Xiaoming Liu, Zhaohan Zhang, Shengchao Liu et al.ACL 2026
