EMNLP Findings2025
Resolving UnderEdit & OverEdit with Iterative & Neighbor-Assisted Model Editing
Bhiman Kumar Baghel, Emma Jordan, Zheyuan Ryan Shi, Xiang Lorraine Li
Abstract
Large Language Models (LLMs) are widely deployed in downstream tasks, but keeping their knowledge up-to-date via retraining or fine-tuning is often computationally expensive. Model editing provides a more efficient alternative by updating a targeted subset of parameters, which often follows the locate-and-edit paradigm. Despite this efficiency, existing methods are limited: edits may fail to inject knowledge (UnderEdit) or unintentionally disrupt unrelated neighboring knowledge (OverEdit). To address these challenges, we propose two complementary methods: iterative model editing, which applies successive edits to mitigate UnderEdit, and neighbor-assisted model editing, which incorporates neighboring knowledge during editing to reduce OverEdit. Our extensive experiments show that these techniques improve editing performance across multiple LLMs, algorithms, and benchmarks, reducing UnderEdit by up to 38 percentage points and OverEdit by up to 6, while remaining broadly applicable to any locate-and-edit method. We release our code at https://github.com/ bhimanbaghel/ResolveUnderOverEdit.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors cites Locating and Editing Factual Associations in GPT
- Locating and Editing Factual Associations in GPT cites Investigating Gender Bias in Language Models Using Causal Mediation Analysis
- WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models cites Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
- WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models cites Locating and Editing Factual Associations in GPT
- PMET: Precise Model Editing in a Transformer cites Locating and Editing Factual Associations in GPT
Builds on17
- Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
- Investigating Gender Bias in Language Models Using Causal Mediation AnalysisJesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian et al.NeurIPS 2020 · 851 citations
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value AdaptorsTom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim et al.NeurIPS 2023 · 349 citations
- PMET: Precise Model Editing in a TransformerXiaopeng Li, Shasha Li, Shezheng Song, Jing Yang et al.AAAI 2024 · 208 citations
- WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language ModelsPeng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu et al.NeurIPS 2024 · 125 citations
Related papers
- Rethinking Residual Distribution in Locate-then-Edit Model EditingXiaopeng Li, Shangwen Wang, Shasha Li, Shezheng Song et al.NeurIPS 2025 · 9 citations
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language ModelsJunfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma et al.ICLR 2025 · 1 citation
- Explainable and Efficient Editing for Large Language ModelsTianyu Zhang, Junfeng Fang, Houcheng Jiang, Baolong Bi et al.WWW 2025 · 8 citations
- Better Call SAUL: Fluent and Consistent Language Model Editing with Generation RegularizationMingyang Wang, Lukas Lange, Heike Adel, Jannik Strötgen et al.EMNLP Findings 2024
- Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMsHaowen Pan, Xiaozhi Wang, Yixin Cao, Zenglin Shi et al.ICLR 2025
