Lune

ICLR2025Top-tier venue

Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs

Haowen Pan, Xiaozhi Wang, Yixin Cao, Zenglin Shi, Xun Yang, Juanzi Li, Meng Wang

Abstract

Knowledge editing aims to update outdated information in Large Language Models (LLMs). A representative line of study is locate-then-edit methods, which typically employ causal tracing to identify the modules responsible for recalling factual knowledge about entities. However, we find these methods are often sensitive only to changes in the subject entity, leaving them less effective at adapting to changes in relations. This limitation results in poor editing locality, which can lead to the persistence of irrelevant or inaccurate facts, ultimately compromising the reliability of LLMs. We believe this issue arises from the insufficient precision of knowledge localization. To address this, we propose a Fine-grained Neuron-level Knowledge Editing (FiNE) method that enhances editing locality without affecting overall success rates. By precisely identifying and modifying specific neurons within feed-forward networks, FiNE significantly improves knowledge localization and editing. Quantitative experiments demonstrate that FiNE efficiently achieves better overall performance compared to existing techniques, providing new insights into the localization and modification of knowledge within LLMs. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

14 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors cites Locating and Editing Factual Associations in GPT
  • Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors cites Memory-Based Model Editing at Scale
  • Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors cites Fast Model Editing at Scale
  • Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors cites Language Models are Few-Shot Learners
  • Locating and Editing Factual Associations in GPT cites Fast Model Editing at Scale
  • Locating and Editing Factual Associations in GPT cites Language Models are Few-Shot Learners
  • Benchmarking and Bridging Emotion Conflicts for Multimodal Emotion Reasoning cites Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization
  • Memory-Based Model Editing at Scale cites Language Models are Few-Shot Learners
  • Fast Model Editing at Scale cites Language Models are Few-Shot Learners
  • MultiAgentESC: A LLM-based Multi-Agent Collaboration Framework for Emotional Support Conversation cites Benchmarking and Bridging Emotion Conflicts for Multimodal Emotion Reasoning
  • MultiAgentESC: A LLM-based Multi-Agent Collaboration Framework for Emotional Support Conversation cites Language Models are Few-Shot Learners
  • Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs cites Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
  • Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs cites Locating and Editing Factual Associations in GPT
  • Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs cites Memory-Based Model Editing at Scale
  • Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs cites Fast Model Editing at Scale
  • Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM Unlearning cites Benchmarking and Bridging Emotion Conflicts for Multimodal Emotion Reasoning
  • Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM Unlearning cites MultiAgentESC: A LLM-based Multi-Agent Collaboration Framework for Emotional Support Conversation
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers14

  1. Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM UnlearningNaixin Zhai, Pengyang Shao, Binbin Zheng, Yonghui Yang et al.ACL 2026 · 10 citations
  2. Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMsJinzhe Liu, Junshu Sun, Shufan Shen, Chenxue Yang et al.NeurIPS 2025 · 8 citations
  3. Benchmarking and Bridging Emotion Conflicts for Multimodal Emotion ReasoningZhiyuan Han, Beier Zhu, Yanlong Xu, Peipei Song et al.ACM MM 2025 · 7 citations
  4. Unsupervised Visual Chain-of-Thought Reasoning via Preference OptimizationKesen Zhao, Beier Zhu, Qianru Sun, Hanwang ZhangICCV 2025 · 3 citations
  5. MultiAgentESC: A LLM-based Multi-Agent Collaboration Framework for Emotional Support ConversationYangyang Xu, Jinpeng Hu, Zhuoer Zhao, Zhangling Duan et al.EMNLP 2025 · 3 citations

Builds on20

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
  3. Fast Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn et al.ICLR 2022 · 527 citations
  4. Memory-Based Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning et al.ICML 2022 · 520 citations
  5. Aging with GRACE: Lifelong Model Editing with Discrete Key-Value AdaptorsTom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim et al.NeurIPS 2023 · 349 citations

Related papers

  1. Editing Memories Through Few Targeted NeuronsWei Zhou, Wei Wei, Guibang Cao, Fei WangAAAI 2025 · 1 citation
  2. Resolving UnderEdit & OverEdit with Iterative & Neighbor-Assisted Model EditingBhiman Kumar Baghel, Emma Jordan, Zheyuan Ryan Shi, Xiang Lorraine LiEMNLP Findings 2025 · 1 citation
  3. GeoEdit: Geometric Knowledge Editing for Large Language ModelsYujie Feng, Li-Ming Zhan, Zexin Lu, Yongxin Xu et al.EMNLP 2025
  4. Explainable and Efficient Editing for Large Language ModelsTianyu Zhang, Junfeng Fang, Houcheng Jiang, Baolong Bi et al.WWW 2025 · 8 citations
  5. Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-TuningTianci Liu, Ruirui Li, Yunzhe Qi, Hui Liu et al.ICLR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines