Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering
Shuliang Liu, Songbo Yang, Dong Fang, Sihang Jia, Yuqi Tang, Lingfeng Su, Ruoshui Peng, Yibo Yan, Xin Zou, Xuming Hu
摘要
Object hallucination critically undermines the reliability of Multimodal Large Language Models, often stemming from a fundamental failure in cognitive introspection, where models blindly trust linguistic priors over specific visual evidence. Existing mitigations remain limited: contrastive decoding approaches operate superficially without rectifying internal semantic misalignments, while current latent steering methods rely on static vectors that lack instance-specific precision. We introduce Vision-Language Introspection (VLI), a training-free inference framework that simulates a metacognitive self-correction process. VLI first performs Attributive Introspection to diagnose hallucination risks via probabilistic conflict detection and localize the causal visual anchors. It then employs Interpretable Bi-Causal Steering to actively modulate the inference process, dynamically isolating visual evidence from background noise while neutralizing blind confidence through adaptive calibration. VLI achieves state-of-the-art performance on advanced models, reducing object hallucination rates by 12.67% on MMHal-Bench and improving accuracy by 5.8% on POPE.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper1
问问它们各自怎么用它它引用的顶会 Paper31
- Evaluating Object Hallucination in Large Vision-Language ModelsYifan Li, Yifan Du, Kun Zhou, Jinpeng Wang 等EMNLP 2023 · 被引用 344 次
- Analyzing and Mitigating Object Hallucination in Large Vision-Language ModelsYiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang 等ICLR 2024 · 被引用 316 次
- HALC: Object Hallucination Reduction via Adaptive Focal-Contrast DecodingZhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao 等ICML 2024 · 被引用 164 次
- Mitigating Object Hallucination via Concentric Causal AttentionYun Xing, Yiheng Li, Ivan Laptev, Shijian LuNeurIPS 2024 · 被引用 78 次
- Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced OptimizationXinyu Lyu, Beitao Chen, Lianli Gao, Hengtao Shen 等NeurIPS 2024 · 被引用 59 次
相关 Paper
- Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMsLiu Yu, Zhonghao Chen, Ping Kuang, Zhikun Feng 等AAAI 2026
- Activation Steering Decoding: Mitigating Hallucination in Large Vision-Language Models through Bidirectional Hidden State InterventionJingran Su, Jingfan Chen, Hongxin Li, Yuntao Chen 等ACL 2025 · 被引用 24 次
- DiVE: Decoupling Intra-layer Visual Evidence for Mitigating Hallucinations in Large Vision-Language ModelsXinwei Li, Li Lin, Hui Jiao, Li Yao 等ACL 2026
- ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMYujun Wang, Aniri, Jinhe Bi, Sören Pirk 等AAAI 2026 · 被引用 27 次
- Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive DecodingSicong Leng, Hang Zhang, Guanzheng Chen, Xin Li 等CVPR 2024
