Lune

EMNLP Findings2025

Enhancing Model Privacy in Federated Learning with Random Masking and Quantization

Zhibo Xu, Jianhao Zhu, Jingwen Xu, Changze Lv, Zhenghua Wang, Zisu Huang, Xiaohua Wang, Muling Wu, Qi Qian, Xiaoqing Zheng, Xuanjing Huang

Abstract

The primary goal of traditional federated learning is to protect data privacy by enabling distributed edge devices to collaboratively train a shared global model while keeping raw data decentralized at local clients. The rise of large language models (LLMs) has introduced new challenges in distributed systems, as their substantial computational requirements and the need for specialized expertise raise critical concerns about protecting intellectual property (IP). This highlights the need for a federated learning approach that can safeguard both sensitive data and proprietary models. To tackle this challenge, we propose FedQSN, a federated learning approach that leverages random masking to obscure a subnetwork of model parameters and applies quantization to the remaining parameters. Consequently, the server transmits only a privacy-preserving proxy of the global model to clients during each communication round, thus enhancing the model's confidentiality. Experimental results across various models and tasks demonstrate that our approach not only maintains strong model performance in federated learning settings but also achieves enhanced protection of model parameters compared to baseline methods. Code and resources are available at https://github.com/zb2313/FedQSN.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • QLoRA: Efficient Finetuning of Quantized LLMs cites SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
  • On the Convergence of FedAvg on Non-IID Data cites Practical Secure Aggregation for Privacy-Preserving Machine Learning
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on10

  1. QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
  2. Practical Secure Aggregation for Privacy-Preserving Machine LearningKallista A. Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone et al.CCS 2017 · 3,936 citations
  3. On the Convergence of FedAvg on Non-IID DataXiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang et al.ICLR 2020 · 2,930 citations
  4. SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language ModelsGuangxuan Xiao, Ji Lin, Mickaël Seznec, Hao Wu et al.ICML 2023 · 1,493 citations
  5. FedScale: Benchmarking Model and System Performance of Federated Learning at ScaleFan Lai, Yinwei Dai, Sanjay Sri Vallabh Singapuram, Jiachen Liu et al.ICML 2022 · 280 citations

Related papers

  1. Promoting Data and Model Privacy in Federated Learning through Quantized LoRAJianhao Zhu, Changze Lv, Xiaohua Wang, Muling Wu et al.EMNLP Findings 2024 · 4 citations
  2. Tunable Soft Prompts are Messengers in Federated LearningChenhe Dong, Yuexiang Xie, Bolin Ding, Ying Shen et al.EMNLP Findings 2023 · 7 citations
  3. FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware FusionTao Fan, Guoqiang Ma, Yuanfeng Song, Lixin Fan et al.ACL 2026
  4. Titanic: Towards Production Federated Learning with Large Language ModelsNingxin Su, Chenghao Hu, Baochun Li, Bo LiINFOCOM 2024 · 30 citations
  5. Safely Learning with Private Data: A Federated Learning Framework for Large Language ModelJiaying Zheng, Hainan Zhang, Lingxiang Wang, Wangjie Qiu et al.EMNLP 2024 · 18 citations
Dusk over the sea between two cliffs drawn in fine vertical lines