Lune

ACL Findings2025

FedLEKE: Federated Locate-then-Edit Knowledge Editing for Multi-Client Collaboration

Zongkai Zhao, Guozeng Xu, Xiuhua Li, Kaiwen Wei, Jiang Zhong

Abstract

Locate-then-Edit Knowledge Editing (LEKE) is a key technique for updating large language models (LLMs) without full retraining. However, existing methods assume a single-user setting and become inefficient in real-world multi-client scenarios, where decentralized organizations (e.g., hospitals, financial institutions) independently update overlapping knowledge, leading to redundant mediator knowledge vector (MKV) computations and privacy concerns. To address these challenges, we introduce Federated Locate-then-Edit Knowledge Editing (FedLEKE), a novel task that enables multiple clients to collaboratively perform LEKE while preserving privacy and reducing computational overhead. To achieve this, we propose FedEdit, a two-stage framework that optimizes MKV selection and reuse. In the first stage, clients locally apply LEKE and upload the computed MKVs. In the second stage, rather than relying solely on server-based MKV sharing, FedLEKE allows clients retrieve relevant MKVs based on cosine similarity, enabling knowledge re-edit and minimizing redundant computations. Experimental results on two benchmark datasets demonstrate that FedEdit retains over 96% of the performance of non-federated LEKE while significantly outperforming a FedAvg-based baseline by approximately twofold. Besides, we find that MEMIT performs more consistently than PMET in the FedLEKE task with our FedEdit framework. Our code is available at https://github.com/zongkaiz/FedLEKE.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • PMET: Precise Model Editing in a Transformer cites Memory-Based Model Editing at Scale
  • PMET: Precise Model Editing in a Transformer cites Editing Large Language Models: Problems, Methods, and Opportunities
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on14

  1. Memory-Based Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning et al.ICML 2022 · 520 citations
  2. Federated Learning from Pre-Trained Models: A Contrastive Learning ApproachYue Tan, Guodong Long, Jie Ma, Lu Liu et al.NeurIPS 2022 · 316 citations
  3. PMET: Precise Model Editing in a TransformerXiaopeng Li, Shasha Li, Shezheng Song, Jing Yang et al.AAAI 2024 · 208 citations
  4. NExT-Chat: An LMM for Chat, Detection and SegmentationAo Zhang, Yuan Yao, Wei Ji, Zhiyuan Liu et al.ICML 2024 · 85 citations
  5. Editing Large Language Models: Problems, Methods, and OpportunitiesYunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng et al.EMNLP 2023 · 83 citations

Related papers

  1. Resolving UnderEdit & OverEdit with Iterative & Neighbor-Assisted Model EditingBhiman Kumar Baghel, Emma Jordan, Zheyuan Ryan Shi, Xiang Lorraine LiEMNLP Findings 2025 · 1 citation
  2. CollabEdit: Towards Non-destructive Collaborative Knowledge EditingJiamu Zheng, Jinghuai Zhang, Tianyu Du, Xuhong Zhang et al.ICLR 2025
  3. Fisher-Driven Adaptive Locating for Knowledge Editing in Large Language ModelsChenghao Xu, Jiexi Yan, Guangtao Lyu, Qi Liu et al.ACL 2026
  4. Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language ModelsJingcheng Deng, Zihao Wei, Liang Pang, Hanxing Ding et al.ICLR 2025
  5. Avoiding Knowledge Edit Skipping in Multi-hop Question Answering with Guided DecompositionYi Liu, Xiangrong Zhu, Xiangyu Liu, Wei Wei et al.EMNLP Findings 2025
Dusk over the sea between two cliffs drawn in fine vertical lines