Batched Contextual Reinforcement
Bangji Yang, Hongbo Ma, Jiajun Fan, Ge Liu
摘要
Large Language Models (LLMs) employing Chain-of-Thought reasoning achieve strong performance but suffer from excessive token consumption that inflates inference costs. Existing efficiency methods—such as explicit length penalties, difficulty estimators, or multi-stage curricula—either degrade reasoning quality or require complex training pipelines. We introduce Batched Contextual Reinforcement (BCR) , a minimalist, single-stage training paradigm that unlocks efficient reasoning through a simple structural modification: training the model to solve N problems simultaneously within a shared context window, rewarded purely by per-instance accuracy. This formulation creates an implicit token budget that yields several key findings: (1) We identify a novel task-scaling law : as the number of concurrent problems N increases at inference time, per-problem token usage decreases monotonically---a phenomenon that arises purely at inference, holds for models both before and after training, and is unrelated to accuracy or the training procedure. BCR makes this regime practical by degrading accuracy far more gracefully than baselines as N grows, establishing N as a controllable throughput dimension. (2) BCR challenges the traditional accuracy-efficiency trade-off by demonstrating a "free lunch" phenomenon at standard single-problem (N=1) inference. Across both 1.5B and 4B model families, BCR reduces token usage by 15.8% to 62.6% while consistently maintaining or improving accuracy across five major mathematical benchmarks (e.g., +13.3% on AIME25 for the 4B model). (3) Qualitative analyses reveal emergent self-regulated efficiency, where models autonomously eliminate redundant metacognitive loops without explicit length supervision. (4) Crucially, we empirically demonstrate that implicit budget constraints successfully circumvent the adversarial gradients and catastrophic optimization collapse inherent to explicit length penalties, offering a highly stable, constraint-based alternative for length control. These results establish BCR as a highly practical framework, demonstrating how simple structural training incentives can unlock latent high-density reasoning modes in LLMs.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper11
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning 等NeurIPS 2023 · 被引用 10,924 次
- Solving Quantitative Reasoning Problems with Language ModelsAitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer 等NeurIPS 2022 · 被引用 2,039 次
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem ComplexityParshin Shojaee, Iman Mirzadeh, Keivan Alizadeh-Vahid, Maxwell Horton 等NeurIPS 2025 · 被引用 507 次
- Reinforcement Learning for Reasoning in Large Language Models with One Training ExampleYiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren 等NeurIPS 2025 · 被引用 314 次
- DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing ReasoningZhiwei He, Tian Liang, Jiahao Xu, Qiuzhi Liu 等ICLR 2026 · 被引用 271 次
相关 Paper
- The Overthinker's DIET: Cutting Token Calories with DIfficulty-AwarE TrainingWeize Chen, Jiarui Yuan, Tailin Jin, Ning Ding 等NeurIPS 2025 · 被引用 13 次
- SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMsDachuan Shi, Abedelkadir Asi, Keying Li, Xiangchi Yuan 等ICLR 2026 · 被引用 22 次
- Overthinking Reduction with Decoupled Rewards and Curriculum Data SchedulingShuyang Jiang, Yusheng Liao, Ya Zhang, Yanfeng Wang 等ICLR 2026 · 被引用 5 次
- Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement LearningHanbing Liu, Lang Cao, Yuanyi Ren, Mengyu Zhou 等ACL 2026 · 被引用 5 次
- SABER: Switchable and Balanced Training for Efficient LLM ReasoningKai Zhao, Yanjun Zhao, Jiaming Song, Shien He 等AAAI 2026 · 被引用 9 次
