CVPR Findings2026
Learning When to Look: A Disentangled Curriculum for Strategic Perception in Multimodal Reasoning
Siqi Yang, Zilve Gao, Haibo Qiu, Fanfan Liu, Peng Shi, Zhixiong Zeng, Qingmin Liao, Lin Ma
摘要
Multimodal Large Language Models (MLLMs) have shown remarkable potential in complex reasoning but remain notoriously brittle during long-chain visual tasks. A critical failure mode is "visual forgetting", where models progressively lose visual grounding as reasoning extends, a phenomenon aptly described as "think longer, see less". We posit this failure stems from current training paradigms prematurely entangling two distinct cognitive skills: (1) abstract logical reasoning ("how-to-think") and (2) strategic visual perception ("when-to-look"). This entanglement leads to a foundational cold-start deficiency and a strategic perception deficit, as models lack an autonomous policy for perception timing. In this paper, we propose a novel curriculum-based framework to disentangle these skills. First, we introduce a disentangled Supervised Fine-Tuning (SFT) curriculum that establishes a robust abstract reasoning backbone on text-only data before anchoring it to vision with a novel Perception-Grounded Chain-of-Thought (PG-CoT) paradigm. Second, we address the strategic perception deficit by formulating timing as a reinforcement learning problem. We design a Pivotal Perception Reward that teaches the model when to look by coupling perceptual actions to linguistic markers of cognitive uncertainty (e.g., "wait", "verify"), thereby cultivating an autonomous grounding policy. Our contributions include the formalization of these two deficiencies and a principled two-stage framework that transforms the model from a heuristic-driven observer into a strategic, grounded reasoner. Project: https://github.com/gaozilve- max/learning-when-to-look.
问 Lune
尽管问,回答会引用原文。
Lune 会读完全文和相关的 Paper,每个回答都附上所依据的原文段落。
引用它的顶会 Paper1
它引用的 Paper28
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 22,562 次引用
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo 等NeurIPS 2022 · 8,168 次引用
- DAPO: An Open-Source LLM Reinforcement Learning System at ScaleQiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan 等NeurIPS 2025 · 2,828 次引用
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual ContextsPan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu 等ICLR 2024 · 1,472 次引用
- MM-Vet: Evaluating Large Multimodal Models for Integrated CapabilitiesWeihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang 等ICML 2024 · 1,191 次引用
相关 Paper
- Thinking in Latent Space: Progressive Multimodal Simplification for Visual ReasoningYuesen Tang, Yiming Yang, Tengfei Bao, Yu TongICML 2026
- Unveiling the Compositional Ability Gap in Vision-Language Reasoning ModelTianle Li, Jihai Zhang, Yongming Rao, Yu ChengNeurIPS 2025 · 17 次引用
- Monet: Reasoning in Latent Visual Space Beyond Image and LanguageQixun Wang, Yang Shi, Yifei Wang, Yuanxing Zhang 等CVPR 2026
- iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement LearningChangbin Zhang, Yujie Zhong, Qiang Zhang, Kai HanICML 2026
- Can Textual Reasoning Improve the Performance of MLLMs on Fine-Grained Visual Classification?Jie Zhu, Yiyang Su, Xiaoming LiuCVPR Findings 2026
