Fisher-Driven Adaptive Locating for Knowledge Editing in Large Language Models
Chenghao Xu, Jiexi Yan, Guangtao Lyu, Qi Liu, Muli Yang, Cheng Deng
Abstract
Large language models (LLMs) store extensive factual knowledge acquired during pretraining, yet this knowledge is inherently static and may become inaccurate or outdated, leading to knowledge hallucinations. Knowledge editing offers an efficient alternative to full retraining by enabling targeted factual updates while preserving overall model behavior. Existing locate-then-edit methods, however, rely on fixed layer selection strategies, treating the locating stage as a static design choice and failing to account for the hierarchical and instancedependent nature of knowledge representation in LLMs. In this paper, we propose FiDAL, a Fisher-driven adaptation-aware locating strategy that dynamically identifies which model components should be edited for a given knowledge update. FiDAL formulates localization as a weight-level decision problem and leverages Fisher Information to select layers that are both influential and sensitive to factual modifications. A lightweight probing stage with low-rank modulation enables efficient localization with minimal overhead. Experiments on standard benchmarks demonstrate that FiDAL consistently improves editing effectiveness and knowledge preservation across multiple editing methods.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 3f5730ed-cddc-4eb2-969e-c4cd24b0cc7cBuilds on19
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Fast Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn et al.ICLR 2022 · 527 citations
- Memory-Based Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning et al.ICML 2022 · 520 citations
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value AdaptorsTom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim et al.NeurIPS 2023 · 349 citations
- PMET: Precise Model Editing in a TransformerXiaopeng Li, Shasha Li, Shezheng Song, Jing Yang et al.AAAI 2024 · 208 citations
Related papers
- Locate-then-edit for Multi-hop Factual Recall under Knowledge EditingZhuoran Zhang, Yongxiang Li, Zijian Kan, Keyuan Cheng et al.ICML 2025
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language ModelsJunfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma et al.ICLR 2025 · 1 citation
- Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMsHaowen Pan, Xiaozhi Wang, Yixin Cao, Zenglin Shi et al.ICLR 2025
- Explainable and Efficient Editing for Large Language ModelsTianyu Zhang, Junfeng Fang, Houcheng Jiang, Baolong Bi et al.WWW 2025 · 8 citations
- CAKE: Causal-Guided Adaptive Knowledge Editing for LLMsShuxin Liu, Jianhao ZhangACL 2026
