Lune

ACL Findings2026

Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs

Byeonggeuk Lim, JungMin Yun, Junehyoung Kwon, Kyeonghyun Kim, YoungBin Kim

Abstract

Large Vision-Language Models (LVLMs) frequently suffer from hallucinations. Existing preference learning-based approaches largely rely on proprietary models to construct preference datasets. We identify that this reliance introduces a distributional mismatch between the proprietary and target models that hinders efficient alignment. To address this, we propose Alignment via VErified Self-correction DPO (AVES-DPO), a framework that aligns LVLMs using in-distribution data derived from the model's intrinsic knowledge. Our approach employs a consensus-based verification mechanism to diagnose diverse hallucinations and guides the model to self-correct, thereby generating preference pairs strictly compatible with its internal distribution. Extensive experiments demonstrate that AVES-DPO surpasses existing baselines in hallucination mitigation while requiring only 5.2k samples.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
  • Evaluating Object Hallucination in Large Vision-Language Models cites Visual Instruction Tuning
  • Aligning Large Multimodal Models with Factually Augmented RLHF cites Evaluating Object Hallucination in Large Vision-Language Models
  • Aligning Large Multimodal Models with Factually Augmented RLHF cites Learning Transferable Visual Models From Natural Language Supervision
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on18

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  3. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  4. Evaluating Object Hallucination in Large Vision-Language ModelsYifan Li, Yifan Du, Kun Zhou, Jinpeng Wang et al.EMNLP 2023 · 344 citations
  5. Aligning Large Multimodal Models with Factually Augmented RLHFZhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu et al.ACL Findings 2024 · 142 citations

Related papers

  1. Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the KeyZhihe Yang, Xufang Luo, Dongqi Han, Yunjian Xu et al.CVPR 2025
  2. V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference OptimizationYuxi Xie, Guanzhen Li, Xiao Xu, Min-Yen KanEMNLP Findings 2024 · 20 citations
  3. Mitigating Hallucinations in Large Vision-Language Models by Self-Injecting HallucinationsYifan Lu, Ziqi Zhang, Chunfeng Yuan, Jun Gao et al.EMNLP Findings 2025 · 1 citation
  4. Stop Learning it all to Mitigate Visual Hallucination, Focus on the Hallucination TargetDokyoon Yoon, Youngsook Song, Woomyoung ParkCVPR 2025
  5. Token Preference Optimization with Self-Calibrated Visual-Anchored Rewards for Hallucination MitigationJihao Gu, Yingyao Wang, Meng Cao, Pi Bu et al.EMNLP Findings 2025 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines