ACL Findings2025
Lifelong Model Editing with Graph-Based External Memory
Yash Kumar Atri, Ahmed M. Alaa, Thomas Hartvigsen
摘要
Large language models (LLMs) have revolutionized natural language processing, yet their practical utility is often limited by persistent issues of hallucinations and outdated parametric knowledge. Although post-training model editing offers a pathway for dynamic updates, existing methods frequently suffer from overfitting and catastrophic forgetting. To tackle these challenges, we propose a novel framework that leverages hyperbolic geometry and graph neural networks for precise and stable model edits. We introduce HYPE 1 , which comprises three key components: (i) Hyperbolic Graph Construction, which uses Poincaré embeddings to represent knowledge triples in hyperbolic space, preserving hierarchical relationships and preventing unintended side effects by ensuring that edits to parent concepts do not inadvertently affect child concepts; (ii) Möbius-Transformed Updates, which apply hyperbolic addition to propagate edits while maintaining structural consistency within the hyperbolic manifold, unlike conventional Euclidean updates that distort relational distances; and (iii) Dual Stabilization, which combines gradient masking and periodic GNN parameter resetting to prevent catastrophic forgetting by focusing updates on critical parameters and preserving long-term knowledge. Experiments on Counter-Fact, CounterFact+, and MQuAKE with GPT-J and GPT2-XL demonstrate that HYPE significantly enhances edit stability, factual accuracy, and multi-hop reasoning.
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
脉络
- Mind the Gap: Assessing Temporal Generalization in Neural Language Models 引用了 Language Models are Few-Shot Learners
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors 引用了 Mind the Gap: Assessing Temporal Generalization in Neural Language Models
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors 引用了 Language Models are Few-Shot Learners
它引用的 Paper24
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah 等NeurIPS 2020 · 64,255 次引用
- Hyperbolic Neural Networks++Ryohei Shimizu, Yusuke Mukuta, Tatsuya HaradaICLR 2021 · 791 次引用
- K-Adapter: Infusing Knowledge into Pre-Trained Models with AdaptersRuize Wang, Duyu Tang, Nan Duan, Zhongyu Wei 等ACL Findings 2021 · 473 次引用
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value AdaptorsTom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim 等NeurIPS 2023 · 349 次引用
- Mind the Gap: Assessing Temporal Generalization in Neural Language ModelsAngeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal 等NeurIPS 2021 · 315 次引用
相关 Paper
- Knowledge Graph Enhanced Large Language Model EditingMengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren 等EMNLP 2024 · 5 次引用
- HyperEdit: Mitigating Hallucinations of Large Language Models via Hyperbolic Representation EditingTongxu Lin, Junping Du, Zhe Xue, Meiyu Liang 等KDD 2026
- Knowledge Graph-Driven Memory Editing with Directional InterventionsJinhu Fu, Kun Wang, Chongye Guo, Junfeng Fang 等EMNLP Findings 2025 · 4 次引用
- HyperPatch: Sequential Knowledge Editing Under n-ary Structural DriftYu-Kai Chan, Wen-Sheng Lien, Dong-Ting Yao, Bo-Kai Ruan 等KDD 2026
- LLM-Based Multi-Hop Question Answering with Knowledge Graph Integration in Evolving EnvironmentsRuirui Chen, Weifeng Jiang, Chengwei Qin, Ishaan Singh Rawal 等EMNLP Findings 2024 · 12 次引用
