Lune

ICLR2025Top-tier venue

Towards Interpreting Visual Information Processing in Vision-Language Models

Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, Fazl Barez

Abstract

Vision-Language Models (VLMs) are powerful tools for processing and understanding text and images. We study the processing of visual tokens in the language model component of LLaVA, a prominent VLM. Our approach focuses on analyzing the localization of object information, the evolution of visual token representations across layers, and the mechanism of integrating visual information for predictions. Through ablation studies, we demonstrated that object identification accuracy drops by over 70% when object-specific tokens are removed. We observed that visual token representations become increasingly interpretable in the vocabulary space across layers, suggesting an alignment with textual tokens corresponding to image content. Finally, we found that the model extracts object information from these refined representations at the last token position for prediction, mirroring the process in text-only language models for factual association tasks. These findings provide crucial insights into how VLMs process and integrate visual information, bridging the gap between our understanding of language and vision models, and paving the way for more interpretable and controllable multimodal systems.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

63 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding cites Visual Instruction Tuning
  • GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution cites Visual Instruction Tuning
  • GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution cites Learning Transferable Visual Models From Natural Language Supervision
  • Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMs cites Visual Instruction Tuning
  • Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMs cites Learning Transferable Visual Models From Natural Language Supervision
  • VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents cites Learning Transferable Visual Models From Natural Language Supervision
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale cites Language Models are Few-Shot Learners
  • Visual Instruction Tuning cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
  • Visual Instruction Tuning cites Language Models are Few-Shot Learners
  • Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
  • BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models cites Language Models are Few-Shot Learners
  • Learning Transferable Visual Models From Natural Language Supervision cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Learning Transferable Visual Models From Natural Language Supervision cites Language Models are Few-Shot Learners
  • Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retention cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retention cites Visual Instruction Tuning
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers63

  1. VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM AgentsKangrui Wang, Pingyue Zhang, Zihan Wang, Yaning Gao et al.NeurIPS 2025 · 66 citations
  2. InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video UnderstandingMinsoo Kim, Kyuhong Shim, Jungwook Choi, Simyung ChangNeurIPS 2025 · 62 citations
  3. Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context RetentionXin Zou, Di Lu, Yizhou Wang, Yibo Yan et al.NeurIPS 2025 · 49 citations
  4. GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K ResolutionFengxiang Wang, Mingshuo Chen, Yueying Li, Di Wang et al.NeurIPS 2025 · 46 citations
  5. Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMsYaniv Nikankin, Dana Arad, Yossi Gandelsman, Yonatan BelinkovNeurIPS 2025 · 37 citations

Builds on29

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  3. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  4. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  5. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations

Related papers

  1. Cross-modal Information Flow in Multimodal Large Language ModelsZhi Zhang, Srishti Yadav, Fengze Han, Ekaterina ShutovaCVPR 2025
  2. Mechanisms of Object Localization in Vision-Language ModelsTimothy Schaumlöffel, Martina G. Vilas, Gemma RoigCVPR 2026 · 1 citation
  3. What's in the Image? A Deep-Dive into the Vision of Vision Language ModelsOmri Kaduri, Shai Bagon, Tali DekelCVPR 2025
  4. Why do LLaVA Vision-Language Models Reply to Images in English?Musashi Hinck, Carolin Holtermann, Matthew L. Olson, Florian Schneider et al.EMNLP Findings 2024 · 5 citations
  5. LLaVA-SP: Enhancing Visual Representation with Visual Spatial Tokens for MLLMsHaoran Lou, Chunxiao Fan, Ziyan Liu, Yuexin Wu et al.ICCV 2025 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines