ACL Findings2025
MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering
Shuo Yang, Caren Han, Siwen Luo, Eduard H. Hovy
摘要
Visual Question Answering (VQA) requires reasoning across visual and textual modalities, yet Large Vision-Language Models (LVLMs) often lack integrated commonsense knowledge, limiting their robustness in real-world scenarios. To address this, we introduce MAGIC-VQA, a novel framework that enhances VQA by systematically integrating commonsense knowledge with LVLMs. MAGIC-VQA employs a three-stage process: (1) Explicit Knowledge Integration from external sources, (2) By-Type Post-Processing for contextual refinement, and (3) Implicit Knowledge Augmentation using a Graph Neural Network (GNN) for structured reasoning. While GNNs bring greater depth to structured inference, they enable superior relational inference beyond LVLMs. MAGIC-VQA bridges a key gap by unifying commonsensse knowledge with LVLMdriven reasoning, eliminating the need for extensive pre-training or complex prompt tuning. Our framework achieves state-of-the-art performance on benchmark datasets, significantly improving commonsense reasoning in VQA. Our implementation is open-sourced on GitHub 1
问 Lune
尽管问,回答会引用原文。
Lune 会读完全文和相关的 Paper,每个回答都附上所依据的原文段落。
引用它的顶会 Paper11
- Semantic-Aware Logical Reasoning via a Semiotic FrameworkYunyao Zhang, Xinglang Zhang, Junxi Sheng, Wenbing Li 等ACL 2026 · 27 次引用
- Omni2Sound: Towards Unified Video-Text-to-Audio Generationyusheng dai, Zehua Chen, Yuxuan Jiang, Qiuhong Ke 等CVPR 2026 · 12 次引用
- TAG-MoE: Task-Aware Gating for Unified Generative Mixture-of-ExpertsYu Xu, Hongbin Yan, Juan Cao, Yiji Cheng 等CVPR 2026 · 7 次引用
- From Where Things Are to What They Are For: Benchmarking Spatial–Functional Intelligence in Multimodal LLMsLe Zhang, Jihan Yang, Soundarya Krishnan, Jimit Majmudar 等CVPR 2026 · 2 次引用
- LongRecipe: Recipe for Efficient Long Context Generalization in Large Language ModelsZhiyuan Hu, Yuliang Liu, Jinman Zhao, Suyuchen Wang 等ACL 2025
它引用的 Paper29
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 47,906 次引用
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 22,562 次引用
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 次引用
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningWenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong 等NeurIPS 2023 · 4,013 次引用
- Align before Fuse: Vision and Language Representation Learning with Momentum DistillationJunnan Li, Ramprasaath R. Selvaraju, Akhilesh Gotmare, Shafiq R. Joty 等NeurIPS 2021 · 2,985 次引用
相关 Paper
- KG-ViP: Bridging Knowledge Grounding and Visual Perception in Multi-modal LLMs for Visual Question AnsweringZhiyang Li, Ao Ke, Yukun Cao, Xike XieACL 2026
- Modality-Aware Integration with Large Language Models for Knowledge-Based Visual Question AnsweringJunnan Dong, Qinggang Zhang, Huachi Zhou, Daochen Zha 等ACL 2024 · 11 次引用
- MM-Reasoner: A Multi-Modal Knowledge-Aware Framework for Knowledge-Based Visual Question AnsweringMahmoud Khademi, Ziyi Yang, Felipe Frujeri, Chenguang ZhuEMNLP Findings 2023 · 4 次引用
- GraphVis: Boosting LLMs with Visual Knowledge Graph IntegrationYihe Deng, Chenchen Ye, Zijie Huang, Mingyu Derek Ma 等NeurIPS 2024 · 23 次引用
- Query and Attention Augmentation for Knowledge-Based Explainable ReasoningYifeng Zhang, Ming Jiang, Qi ZhaoCVPR 2022 · 14 次引用
