Analyzing and Mitigating Object Hallucination: A Training Bias Perspective

As scaling up training data has significantly improved the general multimodal capabilities of Large Vision-Language Models (LVLMs), they still suffer from the hallucination issue, generating text that is inconsistent with the visual input. This phenomenon motivates us to systematically investigate the role of training data in hallucination. We introduce a new benchmark, POPEv2, which consists of counterfactual images collected from the training data of LVLMs with certain objects masked. Through comprehensive evaluation on POPEv2, we find that current LVLMs suffer from training bias: they fail to fully leverage their training data and hallucinate more frequently on images seen during training. Specifically, they perform poorly on counterfactual images, often incorrectly answering "Yes" to questions about masked objects. To understand this issue, we conduct probing experiments on the models' internal components, revealing that this training bias is primarily located in the language modeling (LM) head, which fails to correctly translate accurate visual representations into textual outputs. Based on these findings, we propose Obliviate, an efficient and lightweight unlearning method designed to mitigate object hallucination via training bias unlearning. Obliviate identifies the discrepancy between ground-truth labels and model outputs on the training data as a proxy for bias and adopts a parameter-and data-efficient fine-tuning strategy that only updates the LM head. Extensive experiments demonstrate the effectiveness of our approach. While only reusing the training data and updating approximately 2% of the parameters, Obliviate significantly reduces hallucination across both discriminative and generative tasks. Furthermore, it demonstrates strong scalability with respect to both model size (2B to 72B) and training data volume, and exhibits promising generalization to hallucination types beyond object-level hallucination. Our code and data will be publicly released. * Equal contribution † Corresponding author (b) Target Object Selection and Masking (c) Normal & Counterfactual VQA Qwen2-VL: Yes, there is a laptop in the image. InternVL2: Yes, there is a laptop in the image. The laptop is open and placed on a table. LLaVA-1.5: Yes, there is a laptop in the image, and a woman and a young girl are looking at it. phone (a) Training Images Selection Question: Is there a laptop in the image? + Counterfactual Image person1 person2 laptop Qwen2-VL: Yes, there is a laptop in the image. InternVL2: Yes, the woman in the image is using a laptop to show something on the screen to her daughter. LLaVA-1.5: Yes, there is a laptop featured in the image, with both a girl and a woman sitting in front of it.