EMNLP Findings2025
Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations
Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang
Abstract
Recently, multimodal large language models (MLLMs) have demonstrated remarkable performance in visual-language tasks. However, the authenticity of the responses generated by MLLMs is often compromised by object hallucinations. We identify that a key cause of these hallucinations is the model's over-susceptibility to image frequency features in detecting objects. In this paper, we introduce Multi-Frequency Perturbations (MFP), a simple, costeffective, and pluggable adversarial training method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference, thereby mitigating hallucinations. Experimental results demonstrate that our method significantly mitigates object hallucinations across various model architectures. Furthermore, as a training-time method, MFP can be combined with inference-time methods to achieve state-of-the-art performance on the CHAIR benchmark.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning cites BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
- Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward cites Multi-Frequency Contrastive Decoding: Alleviating Hallucinations for Large Vision-Language Models
- Multi-Frequency Contrastive Decoding: Alleviating Hallucinations for Large Vision-Language Models cites Visual Instruction Tuning
- Multi-Frequency Contrastive Decoding: Alleviating Hallucinations for Large Vision-Language Models cites Learning Transferable Visual Models From Natural Language Supervision
- Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation cites Learning Transferable Visual Models From Natural Language Supervision
- Sigmoid Loss for Language Image Pre-Training cites BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
- Sigmoid Loss for Language Image Pre-Training cites Learning Transferable Visual Models From Natural Language Supervision
- Shallow Focus, Deep Fixes: Enhancing Shallow Layers Vision Attention Sinks to Alleviate Hallucination in LVLMs cites InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
Cited by top-tier papers3
- Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment RewardShizhan Gong, Minda Hu, Qiyuan Zhang, Chen Ma et al.CVPR 2026 · 1 citation
- Multi-Frequency Contrastive Decoding: Alleviating Hallucinations for Large Vision-Language ModelsBingqian Liu, Fu Zhang, Guoqing Chen, Jingwei ChengEMNLP 2025
- Shallow Focus, Deep Fixes: Enhancing Shallow Layers Vision Attention Sinks to Alleviate Hallucination in LVLMsXiaofeng Zhang, Yihao Quan, Chen Shen, Chaochen Gu et al.EMNLP 2025
Builds on28
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 citations
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningWenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong et al.NeurIPS 2023 · 4,013 citations
- Sigmoid Loss for Language Image Pre-TrainingXiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, Lucas BeyerICCV 2023 · 2,932 citations
Related papers
- Poison as Cure: Visual Noise for Mitigating Object Hallucinations in LVMsKejia Zhang, Keda Tao, Jiasheng Tang, Huan WangNeurIPS 2025 · 13 citations
- On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language ModelsHoigi Seo, Dong Un Kang, Hyunjin Cho, Joohoon Lee et al.NeurIPS 2025 · 4 citations
- DHCP: Detecting Hallucinations by Cross-modal Attention Pattern in Large Vision-Language ModelsYudong Zhang, Ruobing Xie, Xingwu Sun, Yiqing Huang et al.ACM MM 2025 · 1 citation
- Alleviating Hallucinations in Large Vision-Language Models via Decoding-Time Perturbation AdaptationJiaqi Bai, Hongcheng Guo, Jiaheng Liu, Zhibo Zhou, Jian Yang, Feiran HuangCVPR Findings 2026
- Evaluating Object Hallucination in Large Vision-Language ModelsYifan Li, Yifan Du, Kun Zhou, Jinpeng Wang et al.EMNLP 2023 · 344 citations
