ACL Findings2026
CaPEdit: Capability-Preserving Lifelong Knowledge Editing For Language Models
Song-Li Wu, Zhaocheng Du, Xianquan Wang, Jingyi Wang
Abstract
Lifelong knowledge editing (LKE) aims to incrementally correct factual inaccuracies in large language models (LLMs), but sequential edits can lead to substantial degradation of capabilities. Existing approaches primarily rely on static parameter regularization, which restricts knowledge integration and fails to prevent cumulative capability degradation. We argue that an important source of this degradation lies in the temporal mismatch between locally editable factual knowledge and procedural knowledge, which is gradually acquired, guides task execution, and cannot be reliably updated by rapid edits. To this end, we formulate LKE as a dual-timescale process, explicitly decoupling fast-updating factual knowledge from slow-evolving procedural knowledge. Based on this formulation, we propose CaPEdit, a framework that preserves model capabilities under LKE. It first synthesizes procedural knowledge across successive edits, and subsequently performs parameter updates guided jointly by factual supervision and the synthesized procedural signal. To ensure stability under long edit sequences, CaPEdit is trained via a hybrid optimization scheme, combining step-wise updates for rapid factual correction with trajectory-level optimization to facilitate gradual procedural adaptation. Experiments demonstrate that CaPEdit improves capability preservation across all fundamental capabilities by 49.78%, achieves superior editing performance, and requires only 18.07% of the editing time of most existing methods.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on23
- TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
- Fast Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn et al.ICLR 2022 · 527 citations
- Memory-Based Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning et al.ICML 2022 · 520 citations
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value AdaptorsTom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim et al.NeurIPS 2023 · 349 citations
- Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP TasksYizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi et al.EMNLP 2022 · 238 citations
Related papers
- Lifelong Knowledge Editing requires Better RegularizationAkshat Gupta, Phudish Prateepamornkul, Maochuan Lu, Ahmed M. Alaa et al.EMNLP Findings 2025 · 1 citation
- EvoEdit: Evolving Null-space Alignment for Robust and Efficient Knowledge EditingSicheng Lyu, Yu Gu, Xinyu Wang, Jerry Huang et al.ACL Findings 2026
- History Matters: Temporal Knowledge Editing in Large Language ModelXunjian Yin, Jin Jiang, Liming Yang, Xiaojun WanAAAI 2024 · 18 citations
- Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language ModelsZihao Lin, Mohammad Beigi, Hongxuan Li, Yufan Zhou et al.ACL 2024 · 1 citation
- LyapLock: Bounded Knowledge Preservation in Sequential Large Language Model EditingPeng Wang, Biyu Zhou, Xuehai Tang, Jizhong Han et al.EMNLP 2025
