Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Models
Minghe Shen, Zhuo Zhi, Chonghan Liu, Shuo Xing, Zhengzhong Tu, Che Liu
摘要
Recent studies posit that Reinforcement Learning with Verifiable Rewards (RLVR) primarily amplifies behaviors inherent to the pre-training distribution rather than inducing new capabilities, but these insights are predominantly limited to language-only domains, leaving the dynamics of visual-centric spatial reasoning under-explored. To examine the impact of RLVR on the capability boundaries of Vision-Language Models (VLMs), we introduce Ariadne, a controlled framework based on synthetic maze navigation where the reasoning difficulty is precisely regulated by path length and the number of turns. We demonstrate that applying RLVR extends the spatial reasoning boundary, achieving success on problems where the base policy VLM consistently attains accuracy despite increasing pass@k sampling budgets, indicating that the optimized policy successfully navigates search spaces that were effectively unreachable by the base distribution. Furthermore, despite being trained exclusively on synthetic mazes, we evaluate the model on two real-world navigation benchmarks (MapBench and ReasonMap) in a zero-shot setting. The observed improvements in these out-of-domain tasks suggest genuine spatial reasoning capability expansion rather than mere sampling efficiency.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper8
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida 等NeurIPS 2022 · 被引用 24,707 次
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang 等NeurIPS 2025 · 被引用 1,109 次
- Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language ModelsWenxuan Huang, Bohan Jia, Shaosheng Cao, Zheyu Ye 等ICLR 2026 · 被引用 670 次
- Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data ContaminationMingqi Wu, Zhihao Zhang, Qiaole Dong, Zhiheng Xi 等AAAI 2026 · 被引用 67 次
- SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement LearningZhongwei Wan, Zhihao Dou, Che Liu, Yu Zhang 等NeurIPS 2025 · 被引用 63 次
相关 Paper
- ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language ModelsZirui Song, Guangxian Ouyang, Mingzhe Li, Yuheng Ji 等AAAI 2026 · 被引用 21 次
- Unlocking Zero-Shot Geospatial Reasoning via Indirect RewardsChenhui Xu, Fuxun Yu, Michael Bianco, Jacob Kovarskiy 等ICML 2026 · 被引用 3 次
- RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy OptimizationYihong Dong, Xue Jiang, Yongding Tao, Huanyu Liu 等ACL 2026 · 被引用 34 次
- Vision-G1: Towards General Reasoning Vision-Language Models via Reinforcement LearningYuheng Zha, Kun Zhou, Yujia Wu, Yushu Wang 等AAAI 2026
- SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMsChanuk Lee, Minki Kang, Sung Ju HwangICML 2026 · 被引用 1 次
