Lune

ACL Findings2025

Lifelong Model Editing with Graph-Based External Memory

Yash Kumar Atri, Ahmed M. Alaa, Thomas Hartvigsen

Abstract

Large language models (LLMs) have revolutionized natural language processing, yet their practical utility is often limited by persistent issues of hallucinations and outdated parametric knowledge. Although post-training model editing offers a pathway for dynamic updates, existing methods frequently suffer from overfitting and catastrophic forgetting. To tackle these challenges, we propose a novel framework that leverages hyperbolic geometry and graph neural networks for precise and stable model edits. We introduce HYPE 1 , which comprises three key components: (i) Hyperbolic Graph Construction, which uses Poincaré embeddings to represent knowledge triples in hyperbolic space, preserving hierarchical relationships and preventing unintended side effects by ensuring that edits to parent concepts do not inadvertently affect child concepts; (ii) Möbius-Transformed Updates, which apply hyperbolic addition to propagate edits while maintaining structural consistency within the hyperbolic manifold, unlike conventional Euclidean updates that distort relational distances; and (iii) Dual Stabilization, which combines gradient masking and periodic GNN parameter resetting to prevent catastrophic forgetting by focusing updates on critical parameters and preserving long-term knowledge. Experiments on Counter-Fact, CounterFact+, and MQuAKE with GPT-J and GPT2-XL demonstrate that HYPE significantly enhances edit stability, factual accuracy, and multi-hop reasoning.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Mind the Gap: Assessing Temporal Generalization in Neural Language Models cites Language Models are Few-Shot Learners
  • Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors cites Mind the Gap: Assessing Temporal Generalization in Neural Language Models
  • Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on24

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Hyperbolic Neural Networks++Ryohei Shimizu, Yusuke Mukuta, Tatsuya HaradaICLR 2021 · 791 citations
  3. K-Adapter: Infusing Knowledge into Pre-Trained Models with AdaptersRuize Wang, Duyu Tang, Nan Duan, Zhongyu Wei et al.ACL Findings 2021 · 473 citations
  4. Aging with GRACE: Lifelong Model Editing with Discrete Key-Value AdaptorsTom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim et al.NeurIPS 2023 · 349 citations
  5. Mind the Gap: Assessing Temporal Generalization in Neural Language ModelsAngeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal et al.NeurIPS 2021 · 315 citations

Related papers

  1. Knowledge Graph Enhanced Large Language Model EditingMengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren et al.EMNLP 2024 · 5 citations
  2. HyperEdit: Mitigating Hallucinations of Large Language Models via Hyperbolic Representation EditingTongxu Lin, Junping Du, Zhe Xue, Meiyu Liang et al.KDD 2026
  3. Knowledge Graph-Driven Memory Editing with Directional InterventionsJinhu Fu, Kun Wang, Chongye Guo, Junfeng Fang et al.EMNLP Findings 2025 · 4 citations
  4. HyperPatch: Sequential Knowledge Editing Under n-ary Structural DriftYu-Kai Chan, Wen-Sheng Lien, Dong-Ting Yao, Bo-Kai Ruan et al.KDD 2026
  5. LLM-Based Multi-Hop Question Answering with Knowledge Graph Integration in Evolving EnvironmentsRuirui Chen, Weifeng Jiang, Chengwei Qin, Ishaan Singh Rawal et al.EMNLP Findings 2024 · 12 citations
Dusk over the sea between two cliffs drawn in fine vertical lines