Lune

ACL Findings2026

Trident: Self-Supervised Preference Alignment via Triplet Regularization

Yingnan Guo, Kejia Chen, Xiaofeng Zhang, Zifei Wu, Yu Zhang

Abstract

Aligning Large Vision-Language Models (LVLMs) to mitigate hallucinations typically relies on high-quality preference data. However, in self-supervised settings, standard binary preference optimization (e.g., DPO) suffers from noisy supervision and semantic ambiguity, as automatically generated chosen responses are not guaranteed to be superior to rejected ones. In this work, we propose Trident, a fully selfsupervised framework that ensures robust alignment via a structured triplet paradigm. Trident autonomously constructs reliable preference triplets-comprising semantically enriched (chosen), degraded (rejected), and neutral (anchor) responses-through automated visual perturbations and self-summarization. We further introduce Trident Preference Regularization (TPR), a novel objective that utilizes an adaptive margin to enforce semantic separation between the triplet components while preventing deviation from the pretrained distribution. Despite requiring no human annotations or external reward models, Trident consistently outperforms state-of-the-art RLHF and RLAIF baselines. For instance, on LLaVA-1.5-7B, it reduces the hallucination rate on AM-BER to 11.3% and achieves 95.70% precision on POPE using only 4k self-generated triplets and a single epoch. This validates structured triplet supervision as a scalable paradigm for robust self-supervised alignment.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning cites BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
  • InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning cites Flamingo: a Visual Language Model for Few-Shot Learning
  • InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning cites MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
  • Flamingo: a Visual Language Model for Few-Shot Learning cites BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
  • MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models cites BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
  • MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models cites Flamingo: a Visual Language Model for Few-Shot Learning
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on17

  1. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  2. Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations
  3. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 citations
  4. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningWenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong et al.NeurIPS 2023 · 4,013 citations
  5. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language ModelsDeyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li et al.ICLR 2024 · 3,079 citations

Related papers

  1. Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMsByeonggeuk Lim, JungMin Yun, Junehyoung Kwon, Kyeonghyun Kim et al.ACL Findings 2026
  2. Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the KeyZhihe Yang, Xufang Luo, Dongqi Han, Yunjian Xu et al.CVPR 2025
  3. Token Preference Optimization with Self-Calibrated Visual-Anchored Rewards for Hallucination MitigationJihao Gu, Yingyao Wang, Meng Cao, Pi Bu et al.EMNLP Findings 2025 · 2 citations
  4. Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference OptimizationShuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai et al.EMNLP 2025 · 2 citations
  5. Self-alignment of Large Video Language Models with Refined Regularized Preference OptimizationPritam Sarkar, Ali EtemadNeurIPS 2025 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines