ACL Findings2025
MEMIT-Merge: Addressing MEMIT's Key-Value Conflicts in Same-Subject Batch Editing for LLMs
Abstract
As large language models continue to scale up, knowledge editing techniques that modify models' internal knowledge without full retraining have gained significant attention. MEMIT, a prominent batch editing algorithm, stands out for its capability to perform mass knowledge modifications. However, we uncover that MEMIT's editing efficacy significantly deteriorates when processing batches containing multiple edits sharing the same subject. Our analysis reveals this stems from MEMIT's key value modeling framework: identical keys (derived from the shared subject) are forced to represent different values (corresponding to different knowledge), resulting in update conflicts during editing. Addressing this issue, we propose MEMIT-Merge, an enhanced approach that merges value computation processes for facts sharing the same subject, effectively resolving the performance degradation in samesubject batch editing scenarios. Experimental results demonstrate that when MEMIT's edit success rate drops to around 50% at larger batch sizes, MEMIT-Merge maintains a success rate exceeding 90%, showcasing remarkable robustness to subject entity collisions. The code is available at https://github.com/NUSTM/ MEMIT-Merge.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Locating and Editing Factual Associations in GPT cites Mass-Editing Memory in a Transformer
- Mass-Editing Memory in a Transformer cites Locating and Editing Factual Associations in GPT
- Locating and Editing Factual Associations in GPT cites Fast Model Editing at Scale
- Conflict-Aware Knowledge Editing in the Wild: Semantic-Augmented Graph Representation for Unstructured Text cites Locating and Editing Factual Associations in GPT
- Conflict-Aware Knowledge Editing in the Wild: Semantic-Augmented Graph Representation for Unstructured Text cites Mass-Editing Memory in a Transformer
- Massive Editing for Large Language Models via Meta Learning cites Locating and Editing Factual Associations in GPT
- Massive Editing for Large Language Models via Meta Learning cites Mass-Editing Memory in a Transformer
- Massive Editing for Large Language Models via Meta Learning cites Fast Model Editing at Scale
- ChainEdit: Propagating Ripple Effects in LLM Knowledge Editing through Logical Rule-Guided Chains cites Locating and Editing Factual Associations in GPT
- ChainEdit: Propagating Ripple Effects in LLM Knowledge Editing through Logical Rule-Guided Chains cites Mass-Editing Memory in a Transformer
- ChainEdit: Propagating Ripple Effects in LLM Knowledge Editing through Logical Rule-Guided Chains cites Fast Model Editing at Scale
- PMET: Precise Model Editing in a Transformer cites Locating and Editing Factual Associations in GPT
- PMET: Precise Model Editing in a Transformer cites Mass-Editing Memory in a Transformer
Cited by top-tier papers2
- Conflict-Aware Knowledge Editing in the Wild: Semantic-Augmented Graph Representation for Unstructured TextZhange Zhang, Zhicheng Geng, Yuqing Ma, Tianbo Wang et al.NeurIPS 2025 · 2 citations
- ChainEdit: Propagating Ripple Effects in LLM Knowledge Editing through Logical Rule-Guided ChainsZilu Dong, Xiangqing Shen, Zinong Yang, Rui XiaACL 2025
Builds on9
- Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
- Fast Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn et al.ICLR 2022 · 527 citations
- PMET: Precise Model Editing in a TransformerXiaopeng Li, Shasha Li, Shezheng Song, Jing Yang et al.AAAI 2024 · 208 citations
- Massive Editing for Large Language Models via Meta LearningChenmien Tan, Ge Zhang, Jie FuICLR 2024 · 68 citations
- Mass-Editing Memory in a TransformerKevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov et al.ICLR 2023 · 52 citations
Related papers
- SeqMMR: Sequential Model Merging and LLM Routing for Enhanced Batched Sequential Knowledge EditingShanbao Qiao, Xuebing Liu, Akshat Gupta, Seung-Hoon NaACL Findings 2025
- CollabEdit: Towards Non-destructive Collaborative Knowledge EditingJiamu Zheng, Jinghuai Zhang, Tianyu Du, Xuhong Zhang et al.ICLR 2025
- EAMET: Robust Massive Model Editing via Embedding Alignment OptimizationYanbo Dai, Zhenlan Ji, Zongjie Li, Shuai WangICLR 2026
- Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMsWenhao Yu, Zhicong Lu, Bo Lv, Fangyin Ma et al.ACL Findings 2026
- Model Editing at Scale leads to Gradual and Catastrophic ForgettingAkshat Gupta, Anurag Rao, Gopala AnumanchipalliACL Findings 2024 · 12 citations
