Revealing and Mitigating Over-Attention in Knowledge Editing
Pinzheng Wang, Zecheng Tang, Keyan Zhou, Juntao Li, Qiaoming Zhu, Min Zhang
Abstract
Large Language Models (LLMs) have demonstrated superior performance across a wide range of tasks, but they still exhibit undesirable errors due to incorrect knowledge learned from the training data. To avoid this, knowledge editing methods emerged to precisely edit the specific model knowledge via efficiently modifying a very small percentage of parameters. However, those methods can lead to the problem of Specificity Failure, where the existing knowledge and capabilities are severely degraded due to editing. Our preliminary indicates that Specificity Failure primarily stems from the model's attention heads assigning excessive attention scores to entities related to the edited knowledge, thereby unduly focusing on specific snippets within the context, which we denote as the Attention Drift phenomenon. To mitigate such Attention Drift issue, we introduce a simple yet effective method Selective Attention Drift Restriction (SADR), which introduces an additional regularization term during the knowledge editing process to restrict changes in the attention weight distribution, thereby preventing undue focus on the edited entity. Experiments on five frequently used strong LLMs demonstrate the effectiveness of our method, where SADR can significantly mitigate Specificity Failure in the predominant knowledge editing tasks.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors cites Locating and Editing Factual Associations in GPT
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors cites Memory-Based Model Editing at Scale
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors cites Fast Model Editing at Scale
- Locating and Editing Factual Associations in GPT cites Fast Model Editing at Scale
- Mitigating Error Accumulation in Knowledge Editing for Multi-Hop Question Answering cites Locating and Editing Factual Associations in GPT
- Mitigating Error Accumulation in Knowledge Editing for Multi-Hop Question Answering cites Fast Model Editing at Scale
- Multi-granularity Temporal Knowledge Editing over Large Language Models cites Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
- Multi-granularity Temporal Knowledge Editing over Large Language Models cites Locating and Editing Factual Associations in GPT
- Multi-granularity Temporal Knowledge Editing over Large Language Models cites Memory-Based Model Editing at Scale
- Multi-granularity Temporal Knowledge Editing over Large Language Models cites Fast Model Editing at Scale
Cited by top-tier papers2
Builds on34
- Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
- PIQA: Reasoning about Physical Commonsense in Natural LanguageYonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao et al.AAAI 2020 · 2,916 citations
- Fast Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn et al.ICLR 2022 · 527 citations
- Memory-Based Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning et al.ICML 2022 · 520 citations
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value AdaptorsTom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim et al.NeurIPS 2023 · 349 citations
Related papers
- AdaEdit: Advancing Continuous Knowledge Editing For Large Language ModelsQi Li, Xiaowen ChuACL 2025
- TamEdit: Trajectory-Aware Meta-Learning for Specificity-Preserving Continual Knowledge EditingShiqiang Tian, Cheng Ding, Qin Chen, Jie Zhou et al.ACL 2026 · 1 citation
- Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMsJinzhe Liu, Junshu Sun, Shufan Shen, Chenxue Yang et al.NeurIPS 2025 · 8 citations
- Model Editing Harms General Abilities of Large Language Models: Regularization to the RescueJia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu et al.EMNLP 2024 · 8 citations
- Knowledge Graph Enhanced Large Language Model EditingMengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren et al.EMNLP 2024 · 5 citations
