SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
Chenzhi Hu, Qinzhe Hu, Yuhang Xu, Junyi Chen, Ruijie Wang, Shengzhong Liu, Jianxin Li, Fan Wu, Guihai Chen
摘要
Large reasoning models (LRMs) like OpenAI o1 and DeepSeek-R1 achieve high accuracy on complex tasks by adopting long chain-of-thought (CoT) reasoning paths. However, the inherent verbosity of these processes frequently results in redundancy and overthinking. To address this issue, existing works leverage Group Relative Policy Optimization (GRPO) to reduce LRM output length, but their static length-reward designs fail to adapt to problem difficulty and response-length distributions, causing over-compression and compromised accuracy. Therefore, we propose SmartThinker , a novel GRPO-based efficient reasoning method with progressive CoT length calibration. SmartThinker makes a two-fold contribution: First, it dynamically estimates the optimal length with peak accuracy during training and guides overlong responses toward it to reduce reasoning length while sustaining accuracy. Second, it dynamically modulates the length-reward coefficient to avoid the unwarranted penalization of correct reasoning paths. Extensive experimental results show that SmartThinker achieves up to 52.6% length compression with improved accuracy and achieves up to 16.6% accuracy relative improvement on challenging benchmarks like AIME25. The source code can be found at https://github.com/SJTU-RTEAS/SmartThinker.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper15
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida 等NeurIPS 2022 · 被引用 24,707 次
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 被引用 22,562 次
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning 等NeurIPS 2023 · 被引用 10,924 次
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou 等ICLR 2021 · 被引用 7,905 次
- DAPO: An Open-Source LLM Reinforcement Learning System at ScaleQiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan 等NeurIPS 2025 · 被引用 2,828 次
相关 Paper
- ETR: Entropy Trend Reward for Efficient Chain-of-Thought ReasoningXuan Xiong, Huan Liu, Li Gu, Zhixiang Chi 等ACL 2026 · 被引用 2 次
- S-GRPO: Early Exit via Reinforcement Learning in Reasoning ModelsMuzhi Dai, Chenxu Yang, Qingyi SiNeurIPS 2025 · 被引用 100 次
- Optimizing Length Compression in Large Reasoning ModelsZhengxiang Cheng, Dongping Chen, Mingyang Fu, Tianyi ZhouACL 2026 · 被引用 32 次
- DRPO: Efficient Reasoning via Decoupled Reward Policy OptimizationGang Li, Yan Chen, Ming Lin, Tianbao YangICLR 2026 · 被引用 19 次
- Sample More to Think Less: Group Filtered Policy Optimization for Concise ReasoningVaishnavi Shrivastava, Ahmed Hassan Awadallah, Vidhisha Balachandran, Shivam Garg 等ICLR 2026 · 被引用 85 次
