From "Aha Moments" to Controllable Thinking: Toward Meta-Cognitive Reasoning in LRMs via Decoupled Reasoning and Control
Rui Ha, Rui Pu, Chaozhuo Li, Li Sun, Sen Su
摘要
Large Reasoning Models (LRMs) can exhibit step-by-step reasoning, reflection, and backtracking, but these behaviors are often unregulated, leading to overthinking. As a result, LRMs continue generating redundant reasoning even after reaching high-confidence conclusions. This increases inference cost and latency, limiting practical deployment. The root cause is the absence of an intrinsic mechanism to monitor the reasoning state and decide when to continue, backtrack, or stop. We propose MERA, a meta-cognitive reasoning framework that decouples reasoning from control to enable independent optimization of control strategies. MERA constructs high-quality reasoning-control supervision data via a takeover-based pipeline, and transforms long-horizon traces into structured reasoning-control alternating sequences for training. The model is trained with supervised fine-tuning to internalize the structured separation, and further optimized with Control-Segment Policy Optimization (CSPO), which combines segment-wise GRPO with control masking to focus learning on control segments. Experiments across reasoning benchmarks show that MERA improves both efficiency and accuracy.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper8
- Dynamic Early Exit in Reasoning ModelsChenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu 等ICLR 2026 · 被引用 250 次
- Thinkless: LLM Learns When to ThinkGongfan Fang, Xinyin Ma, Xinchao WangNeurIPS 2025 · 被引用 128 次
- Think Only When You Need with Large Hybrid-Reasoning ModelsLingjie Jiang, Xun Wu, Shaohan Huang, Qingxiu Dong 等NeurIPS 2025 · 被引用 71 次
- Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM ReasoningChen Qian, Dongrui Liu, Haochen Wen, Zhen Bai 等NeurIPS 2025 · 被引用 63 次
- Learning on Large-scale Text-attributed Graphs via Variational InferenceJianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan 等ICLR 2023 · 被引用 25 次
相关 Paper
- Intervene When It Doubts: Conjunction-Guided Interactive ReasoningQianyue Wang, Jinwu Hu, Yaofo Chen, Yufeng Wang 等ICML 2026
- Incentivizing Dual Process Thinking for Efficient Large Language Model ReasoningXiaoxue Cheng, Junyi Li, Zhenduo Zhang, Xinyu Tang 等NeurIPS 2025 · 被引用 25 次
- SuCo: Sufficiency-guided Continuous Adaptive ReasoningJiahao Wang, Bingyu Liang, Chenhao Hu, Longhui Zhang 等ICML 2026
- Efficiently Learning To Reason or Not to Reason: Root-token Policy Optimization for Adaptive ThinkingTaehyeon Kim, Hyunsoo Lee, Youngsoo Jang, Moontae LeeACL 2026
- When Simple Problems Wear Complex Costumes: Improving Efficiency in LRM's Adaptive ReasoningJunnan Ren, Yan Zhang, Qian Chen, Yunhang Shen 等ICML 2026
