Lune

EMNLP Findings2025

STEAM: A Semantic-Level Knowledge Editing Framework for Large Language Models

Geunyeong Jeong, Juoh Sun, Seonghee Lee, Harksoo Kim

Abstract

Large Language Models store extensive factual knowledge acquired during large-scale pretraining. However, this knowledge is inherently static, reflecting only the state of the world at the time of training. Knowledge editing has emerged as a promising solution for updating outdated or incorrect facts without full retraining. However, most existing locate-and-edit methods primarily focus on token-level likelihood optimization without addressing semantic coherence. Our analysis reveals that such edited knowledge is often encoded as isolated residual streams in the model's latent space, distinct from pre-existing knowledge and bypassing natural reasoning process. To address this, we propose STEAM, a semantic-level knowledge editing framework that enhances integration of updated knowledge into the model's knowledge structure. STEAM first identifies target representations as semantic anchors for the updated factual association, then guides the internal representation of the edited fact towards these anchors through an alignment loss during optimization. Experimental results demonstrate that STEAM improves model's ability to reason with edited knowledge and enhances semantic coherence, underscoring the importance of latent-space alignment for reliable and coherent knowledge editing. The code is available at https://github.com/GY-Jeong/STEAM.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Linearity of Relation Decoding in Transformer Language Models cites Locating and Editing Factual Associations in GPT
  • Editing Large Language Models: Problems, Methods, and Opportunities cites Locating and Editing Factual Associations in GPT
  • PMET: Precise Model Editing in a Transformer cites Locating and Editing Factual Associations in GPT
  • PMET: Precise Model Editing in a Transformer cites Editing Large Language Models: Problems, Methods, and Opportunities
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on11

  1. Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
  2. PMET: Precise Model Editing in a TransformerXiaopeng Li, Shasha Li, Shezheng Song, Jing Yang et al.AAAI 2024 · 208 citations
  3. Linearity of Relation Decoding in Transformer Language ModelsEvan Hernandez, Arnab Sen Sharma, Tal Haklay, Kevin Meng et al.ICLR 2024 · 163 citations
  4. How Do Large Language Models Acquire Factual Knowledge During Pretraining?Hoyeon Chang, Jinho Park, Seonghyeon Ye, Sohee Yang et al.NeurIPS 2024 · 124 citations
  5. Editing Large Language Models: Problems, Methods, and OpportunitiesYunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng et al.EMNLP 2023 · 83 citations

Related papers

  1. Knowledge Editing in Language Models via Adapted Direct Preference OptimizationAmit Rozner, Barak Battash, Lior Wolf, Ofir LindenbaumEMNLP Findings 2024 · 1 citation
  2. Fisher-Driven Adaptive Locating for Knowledge Editing in Large Language ModelsChenghao Xu, Jiexi Yan, Guangtao Lyu, Qi Liu et al.ACL 2026
  3. Explainable and Efficient Editing for Large Language ModelsTianyu Zhang, Junfeng Fang, Houcheng Jiang, Baolong Bi et al.WWW 2025 · 8 citations
  4. StepKE: Stepwise Knowledge Editing for Multi-Hop Question AnsweringJaewook Lee, Dahyun Jung, Heuiseok LimEMNLP Findings 2025
  5. Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMsWenhao Yu, Zhicong Lu, Bo Lv, Fangyin Ma et al.ACL Findings 2026
Dusk over the sea between two cliffs drawn in fine vertical lines