What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT
Yunzhen Feng, Julia Kempe, Cheng Zhang, Parag Jain, Anthony Hartshorn
摘要
Large reasoning models (LRMs) spend substantial test-time compute on long chain-of-thought (CoT) traces, but what characterizes an effective CoT remains unclear. While prior work reports gains from lengthening CoTs and increasing review (revisiting earlier steps) via appended wait tokens, recent studies suggest that shorter thinking can outperform longer traces. We therefore conduct a systematic evaluation across ten LRMs on math and scientific reasoning. Contrary to the "longer-is-better" narrative, we find that both naive CoT lengthening and increased review are associated with lower accuracy. As CoT unfolds step by step, token-level metrics can conflate verbosity with process quality. We introduce a graph view of CoT to extract structure and identify a single statistic-the Failed-Step Fraction (FSF), the fraction of steps in abandoned branches-that consistently outpredicts length and review ratio for correctness across models. To probe causality, we design two interventions. First, we rank candidate CoTs by each metric at test time, where FSF yields the largest pass@1 gains; second, we edit CoTs to remove failed branches, which significantly improves accuracy, indicating that failed branches bias subsequent reasoning. Taken together, these results characterize effective CoTs as those that fail less and support structure-aware test-time scaling over indiscriminately generating long CoT.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper8
- Sculptor: Empowering LLMs with Cognitive Agency via Active Context ManagementMo Li, L.H. Xu, Qitai Tan, Long Ma 等ICLR 2026 · 被引用 20 次
- The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?Alexander Hägele, Aryo Pradipta Gema, Henry Sleight, Ethan Perez 等ICLR 2026 · 被引用 10 次
- ReJump: A Tree-Jump Representation for Analyzing and Improving LLM ReasoningYuchen Zeng, Shuibai Zhang, Wonjun Kang, Shutong Wu 等ICML 2026 · 被引用 5 次
- Effective Reasoning Chains Reduce Intrinsic DimensionalityArchiki Prasad, Mandar Joshi, Kenton Lee, Mohit Bansal 等ICML 2026 · 被引用 3 次
- Reasoning about Reasoning: BAPO Bounds on Chain-of-Thought Token Complexity in LLMsKiran Tomlinson, Tobias Schnabel, Adith Swaminathan, Jennifer NevilleICML 2026 · 被引用 2 次
它引用的顶会 Paper8
- Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran 等NeurIPS 2023 · 被引用 5,068 次
- When More is Less: Understanding Chain-of-Thought Length in LLMsYuyang Wu, Yifei Wang, Ziyu Ye, Tianqi Du 等ICLR 2026 · 被引用 225 次
- Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph PropertiesGouki Minegishi, Hiroki Furuta, Takeshi Kojima, Yusuke Iwasawa 等NeurIPS 2025 · 被引用 31 次
- ROSCOE: A Suite of Metrics for Scoring Step-by-Step ReasoningOlga Golovneva, Moya Chen, Spencer Poff, Martin Corredor 等ICLR 2023 · 被引用 28 次
- RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement LearningQianyue Hao, Sibo Li, Jian Yuan, Yong LiICLR 2026 · 被引用 20 次
相关 Paper
- Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMsZhen Xiong, Yujun Cai, Zhecheng Li, Yiwei WangEMNLP 2025
- On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case StudyRiccardo Alberghi, Elizaveta Demyanenko, Luca Biggio, Luca SagliettiNeurIPS 2025 · 被引用 2 次
- ReEfBench: Quantifying the Reasoning Efficiency of LLMsZhizhang Fu, Yuancheng Gu, Chenkai Hu, Hanmeng Liu 等ACL 2026 · 被引用 1 次
- Understanding Chain-of-Thought in LLMs through Information TheoryJean-Francois Ton, Muhammad Faaiz Taufiq, Yang LiuICML 2025
- Causal Sufficiency and Necessity Improves Chain-of-Thought ReasoningXiangning Yu, Zhuohan Wang, Linyi Yang, Haoxuan Li 等NeurIPS 2025 · 被引用 19 次
