CVPR Findings2026
Learning When to Look: A Disentangled Curriculum for Strategic Perception in Multimodal Reasoning
Siqi Yang, Zilve Gao, Haibo Qiu, Fanfan Liu, Peng Shi, Zhixiong Zeng, Qingmin Liao, Lin Ma
Abstract
Multimodal Large Language Models (MLLMs) have shown remarkable potential in complex reasoning but remain notoriously brittle during long-chain visual tasks. A critical failure mode is "visual forgetting", where models progressively lose visual grounding as reasoning extends, a phenomenon aptly described as "think longer, see less". We posit this failure stems from current training paradigms prematurely entangling two distinct cognitive skills: (1) abstract logical reasoning ("how-to-think") and (2) strategic visual perception ("when-to-look"). This entanglement leads to a foundational cold-start deficiency and a strategic perception deficit, as models lack an autonomous policy for perception timing. In this paper, we propose a novel curriculum-based framework to disentangle these skills. First, we introduce a disentangled Supervised Fine-Tuning (SFT) curriculum that establishes a robust abstract reasoning backbone on text-only data before anchoring it to vision with a novel Perception-Grounded Chain-of-Thought (PG-CoT) paradigm. Second, we address the strategic perception deficit by formulating timing as a reinforcement learning problem. We design a Pivotal Perception Reward that teaches the model when to look by coupling perceptual actions to linguistic markers of cognitive uncertainty (e.g., "wait", "verify"), thereby cultivating an autonomous grounding policy. Our contributions include the formalization of these two deficiencies and a principled two-stage framework that transforms the model from a heuristic-driven observer into a strategic, grounded reasoner. Project: https://github.com/gaozilve- max/learning-when-to-look.
Ask Lune
Ask anything. The answer quotes the paper.
Lune reads all of it and the papers around it, then shows you the passages behind every answer.
Lune answers from
the full paper · 1 top-tier paper that cites it · related work
Free to start. No credit card required.
Cited by top-tier papers1
Builds on28
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- DAPO: An Open-Source LLM Reinforcement Learning System at ScaleQiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan et al.NeurIPS 2025 · 2,828 citations
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual ContextsPan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu et al.ICLR 2024 · 1,472 citations
- MM-Vet: Evaluating Large Multimodal Models for Integrated CapabilitiesWeihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang et al.ICML 2024 · 1,191 citations
Related papers
- Thinking in Latent Space: Progressive Multimodal Simplification for Visual ReasoningYuesen Tang, Yiming Yang, Tengfei Bao, Yu TongICML 2026
- Unveiling the Compositional Ability Gap in Vision-Language Reasoning ModelTianle Li, Jihai Zhang, Yongming Rao, Yu ChengNeurIPS 2025 · 17 citations
- Monet: Reasoning in Latent Visual Space Beyond Image and LanguageQixun Wang, Yang Shi, Yifei Wang, Yuanxing Zhang et al.CVPR 2026
- iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement LearningChangbin Zhang, Yujie Zhong, Qiang Zhang, Kai HanICML 2026
- Can Textual Reasoning Improve the Performance of MLLMs on Fine-Grained Visual Classification?Jie Zhu, Yiyang Su, Xiaoming LiuCVPR Findings 2026
