Lune

EMNLP Findings2024

Better Call SAUL: Fluent and Consistent Language Model Editing with Generation Regularization

Mingyang Wang, Lukas Lange, Heike Adel, Jannik Strötgen, Hinrich Schütze

Abstract

To ensure large language models contain up-todate knowledge, they need to be updated regularly. However, model editing is challenging as it might also affect knowledge that is unrelated to the new data. State-of-the-art methods identify parameters associated with specific knowledge and then modify them via direct weight updates. However, these locate-and-edit methods suffer from heavy computational overhead and lack theoretical validation. In contrast, directly fine-tuning the model on requested edits affects the model's behavior on unrelated knowledge, and significantly damages the model's generation fluency and consistency. To address these challenges, we propose SAUL, a streamlined model editing method that uses sentence concatenation with augmented random facts for generation regularization. Evaluations on three model editing benchmarks show that SAUL is a practical and reliable solution for model editing outperforming state-of-theart methods while maintaining generation quality and reducing computational overhead.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models cites Memory-Based Model Editing at Scale
  • Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models cites Fast Model Editing at Scale
  • Fast Model Editing at Scale cites LoRA: Low-Rank Adaptation of Large Language Models
  • Editing Large Language Models: Problems, Methods, and Opportunities cites Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on12

  1. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  2. Fast Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn et al.ICLR 2022 · 527 citations
  3. Memory-Based Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning et al.ICML 2022 · 520 citations
  4. Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language ModelsPeter Hase, Mohit Bansal, Been Kim, Asma GhandehariounNeurIPS 2023 · 307 citations
  5. Editing Large Language Models: Problems, Methods, and OpportunitiesYunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng et al.EMNLP 2023 · 83 citations

Related papers

  1. Resolving UnderEdit & OverEdit with Iterative & Neighbor-Assisted Model EditingBhiman Kumar Baghel, Emma Jordan, Zheyuan Ryan Shi, Xiang Lorraine LiEMNLP Findings 2025 · 1 citation
  2. Model Editing Harms General Abilities of Large Language Models: Regularization to the RescueJia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu et al.EMNLP 2024 · 8 citations
  3. Lifelong Knowledge Editing requires Better RegularizationAkshat Gupta, Phudish Prateepamornkul, Maochuan Lu, Ahmed M. Alaa et al.EMNLP Findings 2025 · 1 citation
  4. STEAM: A Semantic-Level Knowledge Editing Framework for Large Language ModelsGeunyeong Jeong, Juoh Sun, Seonghee Lee, Harksoo KimEMNLP Findings 2025 · 1 citation
  5. Should We Really Edit Language Models? On the Evaluation of Edited Language ModelsQi Li, Xiang Liu, Zhenheng Tang, Peijie Dong et al.NeurIPS 2024 · 25 citations
Dusk over the sea between two cliffs drawn in fine vertical lines