VeriThinker: Learning to Verify Makes Reasoning Model Efficient
Zigeng Chen, Xinyin Ma, Gongfan Fang, Ruonan Yu, Xinchao Wang
摘要
Large Reasoning Models (LRMs) excel at complex tasks using Chain-of-Thought (CoT) reasoning. However, their tendency to overthinking leads to unnecessarily lengthy reasoning chains, dramatically increasing inference costs. To mitigate this issue, we introduce VeriThinker, a novel approach for CoT compression. Unlike conventional methods that fine-tune LRMs directly on the original reasoning task using synthetic concise CoT data, we innovatively fine-tune the model solely through an auxiliary verification task. By training LRMs to accurately verify the correctness of CoT solutions, the LRMs inherently become more discerning about the necessity of subsequent self-reflection steps, thereby effectively suppressing overthinking. Extensive experiments validate that VeriThinker substantially reduces reasoning chain lengths while maintaining or even slightly improving accuracy. When applied to DeepSeek-R1-Distill-Qwen-7B, our approach reduces reasoning tokens on MATH500 from 3790 to 2125 while improving accuracy by 0.8% (94.0% to 94.8%), and on AIME25, tokens decrease from 14321 to 10287 with a 2.1% accuracy gain (38.7% to 40.8%). Additionally, our experiments demonstrate that VeriThinker can also be zero-shot generalized to speculative reasoning. Code is available at https://github.com/czg1225/VeriThinker
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper5
- OptimalThinkingBench: Evaluating Over and Underthinking in LLMsPranjal Aggarwal, Seungone Kim, Jack Lanchantin, Sean Welleck 等ICLR 2026 · 被引用 38 次
- Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent ReasoningYifan Wang, Shiyu Li, Peiming Li, Xiaochen Yang 等ACL 2026 · 被引用 14 次
- Learning to Self-Verify Makes Language Models Better ReasonersYuxin Chen, Yu Wang, Yi Zhang, Ziang Ye 等ICML 2026 · 被引用 12 次
- Your Reasoning Model Knows What Counts: Self-Guided Chain-of-Thought Pruning for Efficient ReasoningZi-Ao Ma, Xian-Ling Mao, Tian Lan, Chen Xu 等ACL 2026
- Stop When Further Reasoning Won’t Help: Attention-State Adaptive Generation in Reasoning ModelsJiakai Li, KE QIN, Rongzheng Wang, Yizhuo Ma 等ICML 2026
它引用的顶会 Paper32
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 被引用 22,562 次
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 被引用 18,833 次
- Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu 等ICLR 2022 · 被引用 4,966 次
- Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards 等ICLR 2024 · 被引用 3,045 次
- Fast Inference from Transformers via Speculative DecodingYaniv Leviathan, Matan Kalman, Yossi MatiasICML 2023 · 被引用 1,472 次
相关 Paper
- ConPress: Learning Efficient Reasoning from Multi-Question Contextual PressureJie Deng, Shining Liang, Jun Li, Hongzhi Li 等ICML 2026 · 被引用 3 次
- ThoughtFold: Folding Reasoning Chains via Introspective Preference LearningZiyan Liu, Xueda Shen, Yuzhe Gu, Songyang Gao 等ICML 2026 · 被引用 3 次
- TrimR: Verifier-based Training-Free Thinking Trimming for Efficient Test-Time ScalingWeizhe Lin, Xing Li 023, Zhiyuan Yang, Xiaojin Fu 等ICLR 2026 · 被引用 14 次
- Think Better, Not Longer: Token-Level Marginal Utility for Efficient Reasoning in Large Reasoning ModelsJiawei Li, Yang Gao, Huashan Sun, Chong FengACL 2026
- TokenSqueeze: Performance-Preserving Compression for Reasoning LLMsYuxiang Zhang, Zhengxu Yu, Weihang Pan, Zhongming Jin 等NeurIPS 2025 · 被引用 6 次
