ACL Findings2024
Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models
Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, Jun Zhao
Abstract
Recently, retrieval augmentation and tool augmentation have demonstrated a remarkable capability to expand the internal memory boundaries of language models (LMs) by providing external context. However, internal memory and external context inevitably clash, leading to knowledge conflicts within LMs. In this paper, we aim to interpret the mechanism of knowledge conflicts through the lens of information flow, and then mitigate conflicts by precise interventions at the pivotal point. We find there are some attention heads with opposite effects in the later layers, where memory heads can recall knowledge from internal memory, and context heads can retrieve knowledge from external context. Moreover, we reveal that the pivotal point at which knowledge conflicts emerge in LMs is the integration of inconsistent information flows by memory heads and context heads. Inspired by the insights, we propose a novel method called Pruning Head via PatH PatcHing (PH3), which can efficiently mitigate knowledge conflicts by pruning conflicting attention heads without updating model parameters. PH3 can flexibly control eight LMs to use internal memory (↑ 44.0%) or external context (↑ 38.5%). Moreover, PH3 can also improve the performance of LMs on open-domain QA tasks. We also conduct extensive experiments to demonstrate the cross-model, cross-relation, and cross-format generalization of our method. Our code is publicly available at GitHub 1 .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
48 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts cites Locating and Editing Factual Associations in GPT
- The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation cites Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts
- The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation cites Knowledge Conflicts for LLMs: A Survey
- The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Locating and Editing Factual Associations in GPT cites Investigating Gender Bias in Language Models Using Causal Mediation Analysis
- Knowledge Conflicts for LLMs: A Survey cites Locating and Editing Factual Associations in GPT
- Knowledge Conflicts for LLMs: A Survey cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs cites Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
- Knowledge Circuits in Pretrained Transformers cites Locating and Editing Factual Associations in GPT
- Knowledge Circuits in Pretrained Transformers cites Investigating Gender Bias in Language Models Using Causal Mediation Analysis
Cited by top-tier papers48
- Knowledge Circuits in Pretrained TransformersYunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang et al.NeurIPS 2024 · 71 citations
- Knowledge Conflicts for LLMs: A SurveyRongwu Xu, Zehan Qi, Zhijiang Guo, Cunxiang Wang et al.EMNLP 2024 · 38 citations
- RULE: Reinforcement UnLEarning Achieves Forget-retain Pareto OptimalityChenlong Zhang, Zhuoran Jin, Hongbang Yuan, Jiaheng Wei et al.NeurIPS 2025 · 15 citations
- The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval AugmentationPatrick Kahardipraja, Reduan Achtibat, Thomas Wiegand, Wojciech Samek et al.NeurIPS 2025 · 13 citations
- Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMsYinan Zhong, Qianhao Miao, Yanjiao Chen, Jiangyi Deng et al.NDSS 2026 · 13 citations
Builds on28
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
- Pythia: A Suite for Analyzing Large Language Models Across Training and ScalingStella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley et al.ICML 2023 · 1,822 citations
- Investigating Gender Bias in Language Models Using Causal Mediation AnalysisJesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian et al.NeurIPS 2020 · 851 citations
- Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge ConflictsJian Xie, Kai Zhang, Jiangjie Chen, Renze Lou et al.ICLR 2024 · 294 citations
Related papers
- Conflict-Aware Soft Prompting for Retrieval-Augmented GenerationEunseong Choi, June Park, Hyeri Lee, Jongwuk LeeEMNLP 2025 · 1 citation
- TruthfulRAG: Resolving Factual-level Conflicts in Retrieval-Augmented Generation with Knowledge GraphsShuyi Liu, Yu-Ming Shang, Xi ZhangAAAI 2026 · 2 citations
- FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented GenerationQinggang Zhang, Zhishang Xiang, Yilin Xiao, Le Wang et al.ACL 2025 · 18 citations
- Micro-Act: Mitigate Knowledge Conflict in Question Answering via Actionable Self-ReasoningNan Huo, Jinyang Li, Bowen Qin, Ge Qu et al.ACL 2025
- Enhancing Retrieval-Augmented Large Vision Language Models via Knowledge Conflict MitigationWenbin An, Jiahao Nie, Feng Tian, Mingxiang Cai et al.AAAI 2026
