ReEfBench: Quantifying the Reasoning Efficiency of LLMs
Zhizhang Fu, Yuancheng Gu, Chenkai Hu, Hanmeng Liu, Yue Zhang
摘要
Test-time scaling has enabled Large Language Models (LLMs) to tackle complex reasoning, yet the limitations of current Chain-of-Thought (CoT) evaluation obscures whether performance gains stem from genuine reasoning or mere verbosity. To address this, (1) we propose a novel neuro-symbolic framework for the non-intrusive, comprehensive process-centric evaluation of reasoning. (2) Through this lens, we identify four distinct behavioral prototypes and diagnose the failure modes. (3) We examine the impact of inference mode, training strategy, and model scale. Our analysis reveals that extended token generation is not a prerequisite for deep reasoning. Furthermore, we reveal critical constraints: mixing long and short CoT data in training risks in premature saturation and collapse, while distillation into smaller models captures behavioral length but fails to replicate logical efficacy due to intrinsic capacity limits.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper10
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 被引用 22,562 次
- Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD ExamplesAbulhair Saparov, Richard Yuanzhe Pang, Vishakh Padmakumar, Nitish Joshi 等NeurIPS 2023 · 被引用 145 次
- Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-ThoughtAbulhair Saparov, He HeICLR 2023 · 被引用 38 次
- LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic ProversTheo Olausson, Alex Gu, Benjamin Lipkin, Cedegao E. Zhang 等EMNLP 2023 · 被引用 37 次
- ROSCOE: A Suite of Metrics for Scoring Step-by-Step ReasoningOlga Golovneva, Moya Chen, Spencer Poff, Martin Corredor 等ICLR 2023 · 被引用 28 次
相关 Paper
- Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMsZhen Xiong, Yujun Cai, Zhecheng Li, Yiwei WangEMNLP 2025
- What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoTYunzhen Feng, Julia Kempe, Cheng Zhang, Parag Jain 等ICML 2026
- When More is Less: Understanding Chain-of-Thought Length in LLMsYuyang Wu, Yifei Wang, Ziyu Ye, Tianqi Du 等ICLR 2026 · 被引用 225 次
- Too Long, Do Re-weighting for Efficient LLM Reasoning CompressionZhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji 等ACL 2026 · 被引用 5 次
- Long-Context Reasoning Through Proxy-Based Chain-of-Thought TuningMiao Li, Irina Saparina, Alexander Gurung, Mirella LapataACL 2026
