Lune

ACL Findings2026

IREASONER: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Models

Meghana Sunil, Manikandarajan Venmathimaran, Muthu Subash Kavitha

Abstract

Recent work shows that large multimodal models (LMMs) can self-improve from unlabeled data via self-play and intrinsic feedback. Yet existing self-evolving frameworks mainly reward final outcomes, leaving intermediate reasoning weakly constrained despite its importance for visually grounded decision making. We propose IREASONER, a self-evolving framework that improves an LMM's implicit reasoning by explicitly eliciting chain-of-thought (CoT) and rewarding its internal agreement. In a Proposer-Solver loop over unlabeled images, IREASONER augments outcome-level intrinsic rewards with a trajectory-aware signal defined over intermediate reasoning steps, providing learning signals that distinguish reasoning paths leading to the same answer without ground-truth labels or external judges. Starting from Qwen2.5-VL-7B, IREASONER yields up to +2.1 points across diverse multimodal reasoning benchmarks under fully unsupervised post-training. We hope this work serves as a starting point for reasoning-aware selfimprovement in LMMs in purely unsupervised settings. Our code is available here.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts cites ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning
  • MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts cites Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
  • MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI cites MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
  • MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI cites Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on21

  1. Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question AnsweringPan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu et al.NeurIPS 2022 · 2,727 citations
  2. MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual ContextsPan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu et al.ICLR 2024 · 1,472 citations
  3. Promptbreeder: Self-Referential Self-Improvement via Prompt EvolutionChrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero et al.ICML 2024 · 432 citations
  4. ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical ReasoningAhmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq R. Joty et al.ACL Findings 2022 · 389 citations
  5. MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIXiang Yue, Yuansheng Ni, Tianyu Zheng, Kai Zhang et al.CVPR 2024 · 213 citations

Related papers

  1. VisPlay: Self-Evolving Vision-Language ModelsYicheng He, Chengsong Huang, Zongxia Li, Jiaxin Huang et al.CVPR 2026 · 3 citations
  2. Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM ReasoningKongcheng Zhang, Qi Yao, Shunyu Liu, Yingjie Wang et al.NeurIPS 2025 · 45 citations
  3. First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-TrainingLai Wei, Yuting Li, Chen Wang, Yue Wang et al.NeurIPS 2025 · 28 citations
  4. Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language ReasoningJiaqi Liu, Kaiwen Xiong, Peng Xia, Yiyang Zhou et al.ICML 2026 · 28 citations
  5. See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMsYongchang Zhang, Xianzheng Ma, Tianyi Liu, Guangquan Zhou et al.CVPR 2026 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines