Lune

EMNLP Findings2025

Captioning for Text-Video Retrieval via Dual-Group Direct Preference Optimization

Ji Soo Lee, Byungoh Ko, Jaewon Cho, Howoong Lee, Jaewoon Byun, Hyunwoo J. Kim

Abstract

In text-video retrieval, auxiliary captions are often used to enhance video understanding, bridging the gap between the modalities. While recent advances in multi-modal large language models (MLLMs) have enabled strong zeroshot caption generation, we observe that such captions tend to be generic and indistinguishable across visually similar videos, limiting their utility for fine-grained retrieval. Moreover, conventional captioning approaches are typically evaluated using language generation metrics, such as BLEU, which are not typically tailored for retrieval tasks that require making discriminative distinctions between candidates. To address this, we propose CaRe-DPO, a retrieval framework that directly optimizes caption generation using retrieval relevance scores. At its core is Dual-Group Direct Preference Optimization (DG-DPO), a novel learning strategy that supervises captioning by modeling preferences across groups of distinct video and caption pairs. In addition, we present an MLLM-based retrieval model that incorporates role-embeddings to better distinguish between textual inputs with different functional roles, such as an auxiliary caption and a text query. Through extensive experiments, we demonstrate that CaRe-DPO significantly enhances retrieval performance by effectively leveraging auxiliary knowledge to generate fine-grained captions for retrieval. Code is available at https://github.com/mlvlab/CaReDPO.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
  • DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
  • SimPO: Simple Preference Optimization with a Reference-Free Reward cites Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation
  • SimPO: Simple Preference Optimization with a Reference-Free Reward cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
  • MoE-GRPO: Optimizing Mixture-of-Experts via Reinforcement Learning in Vision-Language Models cites DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO
  • MoE-GRPO: Optimizing Mixture-of-Experts via Reinforcement Learning in Vision-Language Models cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPOJinyoung Park, Jeehye Na, Jinyoung Kim, Hyunwoo J. KimNeurIPS 2025 · 64 citations
  2. MoE-GRPO: Optimizing Mixture-of-Experts via Reinforcement Learning in Vision-Language ModelsDohwan Ko, Jinyoung Park, Seoung Choi, Sanghyeok Lee et al.CVPR 2026 · 3 citations

Builds on24

  1. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  2. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  3. Sigmoid Loss for Language Image Pre-TrainingXiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, Lucas BeyerICCV 2023 · 2,932 citations
  4. SimPO: Simple Preference Optimization with a Reference-Free RewardYu Meng, Mengzhou Xia, Danqi ChenNeurIPS 2024 · 1,203 citations
  5. Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine TranslationHaoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan et al.ICML 2024 · 447 citations

Related papers

  1. Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?Wenhao Wu, Haipeng Luo, Bo Fang, Jingdong Wang et al.CVPR 2023
  2. CaReBench: A Fine-grained Benchmark for Video Captioning and RetrievalYifan Xu, Xinhao Li, Yichun Yang, Desen Meng et al.ICLR 2026 · 10 citations
  3. Bidirectional Likelihood Estimation with Multi-Modal Large Language Models for Text-Video RetrievalDohwan Ko, Ji Soo Lee, Minhyuk Choi, Zihang Meng et al.ICCV 2025 · 4 citations
  4. ViLL-E: Video LLM Embeddings for RetrievalRohit Gupta, Jayakrishnan Unnikrishnan, Fan Fei, Sheng Liu et al.ACL 2026
  5. Attention Grounded Enhancement for Visual Document RetrievalWanqing Cui, Wei Huang, Yazhi Guo, Yibo Hu et al.SIGIR 2026
Dusk over the sea between two cliffs drawn in fine vertical lines