StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning
Hao Wang, Lei Sha, Jie Zhang
摘要
Existing code reasoning methods primarily supervise final code outputs, ignoring intermediate states, often leading to reward hacking where correct answers are obtained through inconsistent reasoning. We propose StepCodeReasoner, a framework that introduces explicit intermediate execution-state supervision. By automatically inserting structured print-based execution-trace anchors into code, the model is trained to predict runtime states at each step, transforming code reasoning into a verifiable, stepwise execution modeling problem. Building on this execution-aware method, we introduce Dual-Granularity GRPO (DG-GRPO), a reinforcement learning algorithm for structured credit assignment at two complementary granularities: inter-trajectory comparison across sampled execution paths and intra-trajectory shaping based on downstream correctness. Extensive experiments demonstrate that StepCodeReasoner achieves state-of-the-art performance in code reasoning. In particular, our 7B model achieves 91.1% on CRUXEval and 86.5% on LiveCodeBench, outperforming the CodeReasoner-7B baseline (86.0% and 77.7%) and GPT-4o (85.6% and 75.1%). Furthermore, on the execution-trace benchmark REval, our model scores 82.9%, outperforming baseline CodeReasoner-7B (72.3%), its 14B counterpart (81.1%), and GPT-4o (77.3%). In addition, StepCodeReasoner improves code generation, achieving 90.1 on HumanEval, 85.0 on MBPP, and 19.4 on LiveCodeBench generation, with an average score of 64.8 versus 62.6 for CodeReasoner-7B.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper10
- DAPO: An Open-Source LLM Reinforcement Learning System at ScaleQiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan 等NeurIPS 2025 · 被引用 2,828 次
- Absolute Zero: Reinforced Self-play Reasoning with Zero DataAndrew Zhao, Yiran Wu, Tong Wu, Quentin Xu 等NeurIPS 2025 · 被引用 361 次
- SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software EvolutionYuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux 等NeurIPS 2025 · 被引用 291 次
- Neural Program Repair with Execution-based BackpropagationHe Ye, Matias Martinez, Martin MonperrusICSE 2022 · 被引用 146 次
- TRACED: Execution-aware Pre-training for Source CodeYangruibo Ding, Benjamin Steenhoek, Kexin Pei, Gail E. Kaiser 等ICSE 2024 · 被引用 29 次
相关 Paper
- ReCode: Reinforcing Code Generation with Reasoning-Process RewardsLishui Fan, Yu Zhang, Mouxiang Chen, Zhongxin LiuACL 2026 · 被引用 19 次
- ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution ReasoningLingxiao Tang, He Ye, Zhaoyang Chu, Muyang Ye 等ACL 2026 · 被引用 2 次
- Smarter Not Harder: Generative Process Evaluation with Intrinsic-Signal Driving and Ability‑Adaptive Reward ShapingTao He, Rongchuan Mu, Lizi Liao, Yixin Cao 等ICLR 2026
- Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language ModelsRunxuan Liu, Xianhao Ou, Xinyan Ma, Jiyuan Wang 等ACL 2026
- Step-GRPO: Enhancing Reasoning Quality and Efficiency via Structured PRM-Based Reinforcement LearningWeijie Li, Jin Wang, Liang-Chih Yu, Xuejie ZhangAAAI 2026 · 被引用 1 次
