Lune

ACL Findings2024

Visual Hallucinations of Multi-modal Large Language Models

Wen Huang, Hongbin Liu, Minxin Guo, Neil Gong

Abstract

Visual hallucination (VH) means that a multi-modal LLM (MLLM) imagines incorrect details about an image in visual question answering. Existing studies find VH instances only in existing image datasets, which results in biased understanding of MLLMs' performance under VH due to limited diversity of such VH instances. In this work, we propose a tool called VHTest to generate a diverse set of VH instances. Specifically, VHTest finds some initial VH instances in existing image datasets (e.g., COCO), generates a text description for each VH mode, and uses a text-to-image generative model (e.g., DALL-E-3) to generate VH images based on the text descriptions. We collect a benchmark dataset with 1,200 VH instances in 8 VH modes using VHTest. We find that existing MLLMs such as GPT-4V, LLaVA-1.5, and MiniGPT-v2 hallucinate for a large fraction of the instances in our benchmark. Moreover, we find that fine-tuning an MLLM using our benchmark dataset reduces its likelihood to hallucinate without sacrificing its performance on other benchmarks. Our benchmarks are publicly available: https://github.com/wenhuang2000/VHTest.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

39 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis cites High-Resolution Image Synthesis with Latent Diffusion Models
  • SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis cites Learning Transferable Visual Models From Natural Language Supervision
  • AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception cites Learning Transferable Visual Models From Natural Language Supervision
  • ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLM cites Evaluating Object Hallucination in Large Vision-Language Models
  • Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models cites Evaluating Object Hallucination in Large Vision-Language Models
  • Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models cites Learning Transferable Visual Models From Natural Language Supervision
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers39

  1. Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language ModelsWeihao Ye, Qiong Wu, Wenhao Lin, Yiyi ZhouAAAI 2025 · 99 citations
  2. Vision Language Models are BiasedAn Vo, Khai-Nguyen Nguyen, Mohammad Reza Taesiri, Thi Tuong Vy Dang et al.ICLR 2026 · 68 citations
  3. CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based VerificationYuchen Tian, Weixiang Yan, Qian Yang, Xuandong Zhao et al.AAAI 2025 · 41 citations
  4. AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics PerceptionYipo Huang, Xiangfei Sheng, Zhichao Yang, Quan Yuan et al.ACM MM 2024 · 34 citations
  5. ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMYujun Wang, Aniri, Jinhe Bi, Sören Pirk et al.AAAI 2026 · 27 citations

Builds on10

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
  3. SDXL: Improving Latent Diffusion Models for High-Resolution Image SynthesisDustin Podell, Zion English, Kyle Lacey, Andreas Blattmann et al.ICLR 2024 · 4,569 citations
  4. Evaluating Object Hallucination in Large Vision-Language ModelsYifan Li, Yifan Du, Kun Zhou, Jinpeng Wang et al.EMNLP 2023 · 344 citations
  5. VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image CaptioningJun Chen, Han Guo, Kai Yi, Boyang Li et al.CVPR 2022 · 169 citations

Related papers

  1. PhD: A ChatGPT-Prompted Visual Hallucination Evaluation DatasetJiazhen Liu, Yuhan Fu, Ruobing Xie, Runquan Xie et al.CVPR 2025
  2. AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language ModelsXiyang Wu, Tianrui Guan, Dianqi Li, Shuaiyi Huang et al.EMNLP Findings 2024 · 18 citations
  3. Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed InputsPeng Ding, Jingyu Wu, Jun Kuang, Dan Ma et al.ACM MM 2024 · 8 citations
  4. HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction DataQifan Yu, Juncheng Li, Longhui Wei, Liang Pang et al.CVPR 2024 · 29 citations
  5. Seeing Is Believing: Rich-Context Hallucination Detection for MLLMs via Backward Visual GroundingPinxue Guo, Chongruo Wu, Xinyu Zhou, Lingyi Hong et al.AAAI 2026
Dusk over the sea between two cliffs drawn in fine vertical lines