Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures
Yi Hu, Jiaqi Gu, Ruxin Wang, Zijun Yao, Hao Peng, Xiaobao Wu, Jianhui Chen, Muhan Zhang, Liangming Pan
摘要
Reinforcement learning (RL) has catalyzed the emergence of Large Reasoning Models (LRMs) that have pushed reasoning capabilities to new heights. While their performance has garnered significant excitement, exploring the internal mechanisms driving these behaviors has become an equally critical research frontier. This paper provides a comprehensive survey of the mechanistic understanding of LRMs, organizing recent findings into three core dimensions: 1) training dynamics, 2) reasoning mechanisms, and 3) unintended behaviors. By synthesizing these insights, we aim to bridge the gap between black-box performance and mechanistic transparency. Finally, we discuss under-explored challenges to outline a roadmap for future mechanistic studies, including the need for applied interpretability, improved methodologies, and a unified theoretical framework.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper1
问问它们各自怎么用它它引用的顶会 Paper26
- ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language ModelsMingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu 等NeurIPS 2025 · 被引用 181 次
- Chain-of-Thought Reasoning In The Wild Is Not Always FaithfulIván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan 等ICML 2026 · 被引用 175 次
- Towards Thinking-Optimal Scaling of Test-Time Compute for LLM ReasoningWenkai Yang, Shuming Ma, Yankai Lin, Furu WeiNeurIPS 2025 · 被引用 141 次
- Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest QuestionsLu Ma, Hao Liang, Meiyi Qiang, Lexiang Tang 等ICLR 2026 · 被引用 103 次
- On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic WeightingWenhao Zhang, Yuexiang Xie, Yuchang Sun, Yanxi Chen 等ICLR 2026 · 被引用 100 次
相关 Paper
- Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language ModelsDan Shi, Zhuowen Han, Simon Ostermann, Renren Jin 等ACL 2026 · 被引用 1 次
- ReJump: A Tree-Jump Representation for Analyzing and Improving LLM ReasoningYuchen Zeng, Shuibai Zhang, Wonjun Kang, Shutong Wu 等ICML 2026 · 被引用 5 次
- Is PRM Necessary? Problem-Solving RL Implicitly Induces PRM Capability in LLMsZhangyin Feng, Qianglong Chen, Ning Lu, Yongqian Li 等NeurIPS 2025 · 被引用 16 次
- Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to InterventionShuochen Chang, Tong Bai, Xiaofeng Zhang, Qianli Ma 等ACL 2026 · 被引用 1 次
- ReSearch: Learning to Reason with Search for LLMs via Reinforcement LearningMingyang Chen, Linzhuang Sun, Tianpeng Li, Haoze Sun 等NeurIPS 2025 · 被引用 125 次
