Fisher-Driven Adaptive Locating for Knowledge Editing in Large Language Models
Chenghao Xu, Jiexi Yan, Guangtao Lyu, Qi Liu, Muli Yang, Cheng Deng
摘要
Large language models (LLMs) store extensive factual knowledge acquired during pretraining, yet this knowledge is inherently static and may become inaccurate or outdated, leading to knowledge hallucinations. Knowledge editing offers an efficient alternative to full retraining by enabling targeted factual updates while preserving overall model behavior. Existing locate-then-edit methods, however, rely on fixed layer selection strategies, treating the locating stage as a static design choice and failing to account for the hierarchical and instancedependent nature of knowledge representation in LLMs. In this paper, we propose FiDAL, a Fisher-driven adaptation-aware locating strategy that dynamically identifies which model components should be edited for a given knowledge update. FiDAL formulates localization as a weight-level decision problem and leverages Fisher Information to select layers that are both influential and sensitive to factual modifications. A lightweight probing stage with low-rank modulation enables efficient localization with minimal overhead. Experiments on standard benchmarks demonstrate that FiDAL consistently improves editing effectiveness and knowledge preservation across multiple editing methods.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper19
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah 等NeurIPS 2020 · 被引用 64,255 次
- Fast Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn 等ICLR 2022 · 被引用 527 次
- Memory-Based Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning 等ICML 2022 · 被引用 520 次
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value AdaptorsTom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim 等NeurIPS 2023 · 被引用 349 次
- PMET: Precise Model Editing in a TransformerXiaopeng Li, Shasha Li, Shezheng Song, Jing Yang 等AAAI 2024 · 被引用 208 次
相关 Paper
- Locate-then-edit for Multi-hop Factual Recall under Knowledge EditingZhuoran Zhang, Yongxiang Li, Zijian Kan, Keyuan Cheng 等ICML 2025
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language ModelsJunfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma 等ICLR 2025 · 被引用 1 次
- Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMsHaowen Pan, Xiaozhi Wang, Yixin Cao, Zenglin Shi 等ICLR 2025
- Explainable and Efficient Editing for Large Language ModelsTianyu Zhang, Junfeng Fang, Houcheng Jiang, Baolong Bi 等WWW 2025 · 被引用 8 次
- CAKE: Causal-Guided Adaptive Knowledge Editing for LLMsShuxin Liu, Jianhao ZhangACL 2026
