Lune

ACL Findings2026

SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis

Zijian Wu, Jinjie Ni, Xiangyan Liu, Zichen Liu, Hang Yan, Michael Qizhe Shieh

Abstract

Vision-language models (VLMs) trained via reinforcement learning with verifiable reward (RLVR) have shown notable progress in scaling test-time compute effectively. In this work, we investigate how synthesized RL data can further improve RLVR. To this end, we propose Syn-thRL-a scalable and guaranteed pipeline for automatic data scaling in reasoning-oriented RL training. SynthRL comprises three key stages: (1) selecting seed questions with appropriate distribution, (2) augmenting them into more challenging variants while preserving the original answers, and (3) a guaranteed verification stage that ensures near-perfect correctness and difficulty enhancement. Our empirical experiments demonstrate SynthRL's scalability and effectiveness. When applied to the MMK12 dataset, SynthRL synthesizes over 3.3K additional verifiable, challenging questions from approximately 8K seed samples. Models trained with our synthesized data achieve consistent gains across five outof-domain visual math reasoning benchmarks, with a significant improvement over baseline models trained on seed data alone. Notably, detailed analysis reveals that the gains are more pronounced on the most challenging evaluation samples, highlighting SynthRL's effectiveness in eliciting deeper and more complex reasoning patterns.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Counteracting the Matthew Effect in Self-Improvement of LVLMs through Head-Tail Re-balancing cites Efficient Memory Management for Large Language Model Serving with PagedAttention
  • MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts cites Flamingo: a Visual Language Model for Few-Shot Learning
  • Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware Decoding cites MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware DecodingZhongxing Xu, Zhonghua Wang, Zhe Qian, Dachuan Shi et al.CVPR 2026 · 16 citations
  2. Counteracting the Matthew Effect in Self-Improvement of LVLMs through Head-Tail Re-balancingXin Guo, Zhiheng Xi, Yiwen Ding, Yitao Zhai et al.ACL 2026 · 1 citation
  3. A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and SolutionsZhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo et al.ACL 2026

Builds on27

  1. Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations
  2. DAPO: An Open-Source LLM Reinforcement Learning System at ScaleQiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan et al.NeurIPS 2025 · 2,828 citations
  3. MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual ContextsPan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu et al.ICLR 2024 · 1,472 citations
  4. WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex InstructionsCan Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng et al.ICLR 2024 · 1,206 citations
  5. Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng et al.SOSP 2023 · 1,016 citations

Related papers

  1. Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive DomainsZhonghang Yuan, Zhefan Wang, Fang Hu, Zihong Chen et al.ACL 2026
  2. ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data SynthesisCongzhi Zhang, Zhibin Wang, Yinchao Ma, Jiawei Peng et al.ICLR 2026 · 24 citations
  3. Long Grounded Thoughts: Synthesizing Visual Problems and Reasoning Chains at ScaleDavid Acuna, Chao-Han Huck Yang, Yuntian Deng, Jaehun Jung et al.ICML 2026 · 1 citation
  4. Beyond Pass@ 1: Self-Play with Variational Problem Synthesis Sustains RLVRXiao Liang, Zhong-Zhi Li, Yeyun Gong, Yelong Shen et al.ICLR 2026 · 57 citations
  5. SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and BeyondJunteng Liu, Yuanxiang Fan, Zhuo Jiang, Han Ding et al.NeurIPS 2025 · 49 citations
Dusk over the sea between two cliffs drawn in fine vertical lines