Lune

ACL Findings2025

Context-DPO: Aligning Language Models for Context-Faithfulness

Baolong Bi, Shaohan Huang, Yiwei Wang, Tianchi Yang, Zihan Zhang, Haizhen Huang, Lingrui Mei, Junfeng Fang, Zehao Li, Furu Wei, Weiwei Deng, Feng Sun

Abstract

Reliable responses from large language models (LLMs) require adherence to user instructions and retrieved information. While alignment techniques help LLMs align with human intentions and values, improving contextfaithfulness through alignment remains underexplored. To address this, we propose Context-DPO, the first alignment method specifically designed to enhance LLMs' contextfaithfulness. We introduce ConFiQA, a benchmark that simulates Retrieval-Augmented Generation (RAG) scenarios with knowledge conflicts to evaluate context-faithfulness. By leveraging faithful and stubborn responses to questions with provided context from ConFiQA, our Context-DPO aligns LLMs through direct preference optimization. Extensive experiments demonstrate that our Context-DPO significantly improves context-faithfulness, achieving 35% to 280% improvements on popular opensource models. Further analysis demonstrates that Context-DPO preserves LLMs' generative capabilities while providing interpretable insights into context utilization.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

17 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models cites Active Retrieval Augmented Generation
  • FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement cites Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models
  • DVP-MVS: Synergize Depth-Edge and Visibility Prior for Multi-View Stereo cites Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models
  • TruthfulQA: Measuring How Models Mimic Human Falsehoods cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • MSP-MVS: Multi-Granularity Segmentation Prior Guided Multi-View Stereo cites Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models
  • Active Retrieval Augmented Generation cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Active Retrieval Augmented Generation cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers17

  1. Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language ModelsBaolong Bi, Shenghua Liu, Yiwei Wang, Yilong Xu et al.ICLR 2026 · 47 citations
  2. MSP-MVS: Multi-Granularity Segmentation Prior Guided Multi-View StereoZhenlong Yuan, Cong Liu, Fei Shen, Zhaoxin Li et al.AAAI 2025 · 22 citations
  3. DVP-MVS: Synergize Depth-Edge and Visibility Prior for Multi-View StereoZhenlong Yuan, Jinguo Luo, Fei Shen, Zhaoxin Li et al.AAAI 2025 · 19 citations
  4. FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented GenerationQinggang Zhang, Zhishang Xiang, Yilin Xiao, Le Wang et al.ACL 2025 · 18 citations
  5. Resisting Contextual Interference in RAG via Parametric-Knowledge ReinforcementChenyu Lin, Yilin Wen, Du Su, Hexiang Tan et al.ICLR 2026 · 13 citations

Builds on24

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  3. TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
  4. Retrieval Augmented Language Model Pre-TrainingKelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat et al.ICML 2020 · 2,937 citations
  5. Active Retrieval Augmented GenerationZhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun et al.EMNLP 2023 · 315 citations

Related papers

  1. RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented GenerationShi-Qi Yan, Quan Liu, Zhen-Hua LingACL 2025 · 4 citations
  2. To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External ContextsYukun Huang, Sanxing Chen, Hongyi Cai, Bhuwan DhingraICLR 2025
  3. FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"Yifei Ming, Senthil Purushwalkam, Shrey Pandit, Zixuan Ke et al.ICLR 2025
  4. Copy-Paste to Mitigate Large Language Model HallucinationsYongchao Long, Yingying Zhang, Xianbin Wen, Xian Wu et al.ICLR 2026 · 2 citations
  5. ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented GenerationPengcheng Huang, Zhenghao Liu, Yukun Yan, Haiyan Zhao et al.NeurIPS 2025 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines