ACL Findings2026
Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs
Byeonggeuk Lim, JungMin Yun, Junehyoung Kwon, Kyeonghyun Kim, YoungBin Kim
摘要
Large Vision-Language Models (LVLMs) frequently suffer from hallucinations. Existing preference learning-based approaches largely rely on proprietary models to construct preference datasets. We identify that this reliance introduces a distributional mismatch between the proprietary and target models that hinders efficient alignment. To address this, we propose Alignment via VErified Self-correction DPO (AVES-DPO), a framework that aligns LVLMs using in-distribution data derived from the model's intrinsic knowledge. Our approach employs a consensus-based verification mechanism to diagnose diverse hallucinations and guides the model to self-correct, thereby generating preference pairs strictly compatible with its internal distribution. Extensive experiments demonstrate that AVES-DPO surpasses existing baselines in hallucination mitigation while requiring only 5.2k samples.
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
脉络
- Visual Instruction Tuning 引用了 Learning Transferable Visual Models From Natural Language Supervision
- Evaluating Object Hallucination in Large Vision-Language Models 引用了 Visual Instruction Tuning
- Aligning Large Multimodal Models with Factually Augmented RLHF 引用了 Evaluating Object Hallucination in Large Vision-Language Models
- Aligning Large Multimodal Models with Factually Augmented RLHF 引用了 Learning Transferable Visual Models From Natural Language Supervision
它引用的 Paper18
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 47,906 次引用
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 次引用
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning 等NeurIPS 2023 · 10,924 次引用
- Evaluating Object Hallucination in Large Vision-Language ModelsYifan Li, Yifan Du, Kun Zhou, Jinpeng Wang 等EMNLP 2023 · 344 次引用
- Aligning Large Multimodal Models with Factually Augmented RLHFZhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu 等ACL Findings 2024 · 142 次引用
相关 Paper
- Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the KeyZhihe Yang, Xufang Luo, Dongqi Han, Yunjian Xu 等CVPR 2025
- V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference OptimizationYuxi Xie, Guanzhen Li, Xiao Xu, Min-Yen KanEMNLP Findings 2024 · 20 次引用
- Mitigating Hallucinations in Large Vision-Language Models by Self-Injecting HallucinationsYifan Lu, Ziqi Zhang, Chunfeng Yuan, Jun Gao 等EMNLP Findings 2025 · 1 次引用
- Stop Learning it all to Mitigate Visual Hallucination, Focus on the Hallucination TargetDokyoon Yoon, Youngsook Song, Woomyoung ParkCVPR 2025
- Token Preference Optimization with Self-Calibrated Visual-Anchored Rewards for Hallucination MitigationJihao Gu, Yingyao Wang, Meng Cao, Pi Bu 等EMNLP Findings 2025 · 2 次引用
