Awakening Visual Reasoning: Mitigating Post-Training Failure in Vision-Text Compression
Xing Xi, Liyao Li, Hao Chen, NINGTAO WANG, Peixian Chen, peilin tong, Xing Fu, Yu Cheng, Haobo Wang, Gang Chen, Junbo Zhao, Ronghua Luo
摘要
Vision-Text Compression (VTC) offers a scalable path for long-context multimodal modeling by rendering textual data into dense visual tokens. While recent Vision-Language Models (VLMs) demonstrate high decoding fidelity (OCR) on such inputs, they exhibit a severe reasoning gap: models that reason robustly on native text often fail on visually compressed equivalents, particularly in long-range retrieval and multi-step deduction. We identify a phenomenon of post-training transfer failure, where standard supervised fine-tuning and reinforcement learning on visual prompts yield marginal gains compared to their textual counterparts. To address this, we propose CoRe (Coordinated Reasoning), a training framework that enforces lockstep consistency between the reasoning processes of textual and visual modalities. By treating the text-conditioned policy as a dynamic anchor, CoRe aligns the visual-conditioned policy via step-wise distribution matching, seamlessly integrating into both SFT and RL pipelines. Extensive evaluations across mathematical reasoning, long-context memory, and tabular retrieval benchmarks show that CoRe significantly outperforms standard visual post-training, recovering up to 70% of the performance gap relative to the textual upper bound and effectively activating latent reasoning capabilities in the compressed visual modality.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper9
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 被引用 22,562 次
- TabFact: A Large-scale Dataset for Table-based Fact VerificationWenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang 等ICLR 2020 · 被引用 674 次
- VPI-Bench: Visual Prompt Injection Attacks for Computer-Use AgentsTri Cao, Bennett Lim, Yue Liu, Yuan Sui 等ICLR 2026 · 被引用 45 次
- Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical ReasoningPan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu 等ICLR 2023 · 被引用 41 次
相关 Paper
- Unveiling the Compositional Ability Gap in Vision-Language Reasoning ModelTianle Li, Jihai Zhang, Yongming Rao, Yu ChengNeurIPS 2025 · 被引用 17 次
- Integrating Visual Interpretation and Linguistic Reasoning for Geometric Problem SolvingZixian Guo, Ming Liu, Qilong Wang, Zhilong Ji 等ICCV 2025 · 被引用 1 次
- Activating Visual Context and Commonsense Reasoning Through Masked Prediction in VLMsJiaao Yu, Shenwei Li, Mingjie Han, Yifei Yin 等AAAI 2026
- Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual ReasoningYana Wei, Liang Zhao, Jianjian Sun, Kangheng Lin 等NeurIPS 2025 · 被引用 39 次
- Scaling RL to Long VideosYukang Chen, Wei Huang, Baifeng Shi, Qinghao Hu 等NeurIPS 2025 · 被引用 91 次
