CVPR Findings2026
CoVCR: Bridging Visual Narrative Gaps via Context Generation for Robust Commonsense Reasoning
Xinyu Li, Shiliang Sun
Abstract
Multimodal large language models (MLLMs) have shown impressive capabilities in visual-language reasoning tasks, largely due to training on datasets where images and texts are well-aligned and semantically rich. However, in realworld scenarios, the visual input may present insufficient narratives-that is, the image may lack the necessary context or detail required to support accurate question answering. This narrative insufficiency limits the model's ability to reason accurately and increases its reliance on superficial correlations. To enhance the robustness of visual reasoning under such conditions, we introduce CoVCR, a contextinfused enhanced framework for visual commonsense reasoning. CoVCR enriches multimodal representations by generating and integrating plausible contextual cues that complement the insufficient visual narrative. It comprises two key components: (1) a visual context generation model, which is pretrained to generate plausible internal (in-scene) and external (temporal) contexts from the image to support downstream reasoning; (2) a context-infused adapter that dynamically extracts and integrates visual features along with both internal and external context. This design enables CoVCR to adaptively reason under varying degrees of visual completeness, yielding more faithful and robust inferences. Experiments show that the proposed method substantially surpasses strong baselines and validates the effectiveness of context-enhanced reasoning.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Visual Instruction Tuning cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
Builds on15
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
- Perceiver IO: A General Architecture for Structured Inputs & OutputsAndrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch et al.ICLR 2022 · 797 citations
Related papers
- MM-Reasoner: A Multi-Modal Knowledge-Aware Framework for Knowledge-Based Visual Question AnsweringMahmoud Khademi, Ziyi Yang, Felipe Frujeri, Chenguang ZhuEMNLP Findings 2023 · 4 citations
- ViCor: Bridging Visual Understanding and Commonsense Reasoning with Large Language ModelsKaiwen Zhou, Kwonjoon Lee, Teruhisa Misu, Xin WangACL Findings 2024 · 7 citations
- Enhancing Advanced Visual Reasoning Ability of Large Language ModelsZhiyuan Li, Dongnan Liu, Chaoyi Zhang, Heng Wang et al.EMNLP 2024 · 10 citations
- Do Vision-Language Transformers Exhibit Visual Commonsense? An Empirical Study of VCRZhenyang Li, Yangyang Guo, Kejie Wang, Xiaolin Chen et al.ACM MM 2023 · 11 citations
- Cantor: Inspiring Multimodal Chain-of-Thought of MLLMTimin Gao, Peixian Chen, Mengdan Zhang, Chaoyou Fu et al.ACM MM 2024 · 20 citations
