EMNLP Findings2025
FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Models
Mingyang Song, Mao Zheng, Zheng Li, Wenjie Yang, Xuan Luo
Abstract
Improving training efficiency continues to be one of the primary challenges in large-scale Reinforcement Learning (RL). In this paper, we investigate how context length and the complexity of training data influence the RL scaling training process of R1-distilled reasoning models, e.g., DeepSeek-R1-Distill-Qwen-1.5B. Our experimental results reveal that: (1) simply controlling the context length and selecting the training data based on the input prompt length can effectively improve the training efficiency of RL scaling, achieving better performance with more concise CoT; (2) properly scaling the context length helps mitigate entropy collapse; and
(3) carefully choosing the context length facilitates achieving efficient LLM training and reasoning. Inspired by these insights, we propose FASTCURL, a curriculum RL framework with stage-wise context scaling to achieve efficient LLM training and reasoning. Extensive experimental results demonstrate that FASTCURL-1.5B-V3 significantly outperforms state-of-theart reasoning models on five competition-level benchmarks and achieves 49.6% accuracy on AIME 2024. Furthermore, FASTCURL-1.5B-Preview surpasses DeepScaleR-1.5B-Preview on five benchmarks while only using a single node with 8 GPUs and a total of 50% of training steps, as shown in Figure 1. The code, training data, and models have been publicly released 1 .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
9 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems cites Solving Quantitative Reasoning Problems with Language Models
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example cites DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example cites OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example cites Solving Quantitative Reasoning Problems with Language Models
- Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL cites DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Solving Quantitative Reasoning Problems with Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- UFT: Unifying Supervised and Reinforcement Fine-Tuning cites DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- RLinf: Flexible and Efficient Large-Scale Reinforcement Learning via Macro-to-Micro Flow Transformation cites DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs cites DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs cites OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
- CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs cites Solving Quantitative Reasoning Problems with Language Models
- When More is Less: Understanding Chain-of-Thought Length in LLMs cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Cited by top-tier papers9
- Reinforcement Learning for Reasoning in Large Language Models with One Training ExampleYiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren et al.NeurIPS 2025 · 314 citations
- Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RLSongjun Tu, Jiahao Lin, Qichao Zhang, Xiangyu Tian et al.NeurIPS 2025 · 69 citations
- RLinf: Flexible and Efficient Large-Scale Reinforcement Learning via Macro-to-Micro Flow TransformationChao Yu, Yuanqing Wang, Zhen Guo, Hao Lin et al.OSDI 2026 · 63 citations
- UFT: Unifying Supervised and Reinforcement Fine-TuningMingyang Liu, Gabriele Farina, Asuman OzdaglarNeurIPS 2025 · 61 citations
- CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMsYongcheng Zeng, Zexu Sun, Bokai Ji, Erxue Min et al.ICLR 2026 · 18 citations
Builds on6
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- DAPO: An Open-Source LLM Reinforcement Learning System at ScaleQiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan et al.NeurIPS 2025 · 2,828 citations
- Solving Quantitative Reasoning Problems with Language ModelsAitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer et al.NeurIPS 2022 · 2,039 citations
- When More is Less: Understanding Chain-of-Thought Length in LLMsYuyang Wu, Yifei Wang, Ziyu Ye, Tianqi Du et al.ICLR 2026 · 225 citations
- OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific ProblemsChaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu et al.ACL 2024 · 18 citations
Related papers
- Scale Down to Speed Up: Dynamic Data Selection for Reinforcement LearningZhuoyue Chen, Jihai Zhang, Ben Liu, Fangquan Lin et al.EMNLP Findings 2025 · 1 citation
- Learn to Reason Efficiently with Adaptive Length-based Reward ShapingWei Liu, Ruochen Zhou, Yiyun Deng, Yuzhen Huang et al.ICLR 2026 · 88 citations
- Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated PenaltyZewei Yu, Lirong Gao, Yuke Zhu, Bo Zheng et al.ICLR 2026 · 2 citations
- How Far Are We from Optimal Reasoning Efficiency?Jiaxuan Gao, Shu Yan, Qixin Tan, Lu Yang et al.NeurIPS 2025 · 12 citations
- AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement LearningYang Chen, Zhuolin Yang, Zihan Liu, Chankyu Lee et al.NeurIPS 2025 · 79 citations
