Lune

ACL Findings2023

Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter

Yi Liu, Xiaohan Bi, Lei Li, Sishuo Chen, Wenkai Yang, Xu Sun

Abstract

Federated Multilingual Neural Machine Translation (Fed-MNMT) has emerged as a promising paradigm for institutions with limited language resources.This approach allows multiple institutions to act as clients and train a unified model through model synchronization, rather than collecting sensitive data for centralized training.This significantly reduces the cost of corpus collection and preserves data privacy.However, as pre-trained language models (PLMs) continue to increase in size, the communication cost for transmitting parameters during synchronization has become a training speed bottleneck.In this paper, we propose a communication-efficient Fed-MNMT framework that addresses this issue by keeping PLMs frozen and only transferring lightweight adapter modules between clients.Since different language pairs exhibit substantial discrepancies in data distributions, adapter parameters of clients may conflict with each other.To tackle this, we explore various clustering strategies to group parameters for integration and mitigate the negative effects of conflicting parameters.Experimental results demonstrate that our framework reduces communication cost by over 98% while achieving similar or even better performance compared to competitive baselines.Further analysis reveals that clustering strategies effectively solve the problem of linguistic discrepancy and pruning adapter modules further improves communication efficiency. 1 lv-pl Global Model de-fr Group 1 Fed-MNMT Our work Group 1 Adapter + -Aggregation -Backbone Model -Adapter modules -Clients -Server Backbone Model de-fr Backbone Model lv-pl Backbone

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Beyond End-to-End: Dynamic Chain Optimization for Private LLM Adaptation on the Edge cites Developmental Federated Tuning: A Cognitive-Inspired Paradigm for Efficient LLM Adaptation
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Perada: Parameter-Efficient Federated Learning Personalization with Generalization GuaranteesChulin Xie, De-An Huang, Wenda Chu, Daguang Xu et al.CVPR 2024 · 15 citations
  2. Beyond End-to-End: Dynamic Chain Optimization for Private LLM Adaptation on the EdgeYebo Wu, Jingguang Li, Chunlin Tian, KaHou Tam et al.ACL 2026 · 1 citation
  3. Developmental Federated Tuning: A Cognitive-Inspired Paradigm for Efficient LLM AdaptationYebo Wu, Jingguang Li, Zhijiang Guo, Li LiICLR 2026

Builds on5

  1. Compacter: Efficient Low-Rank Hypercomplex Adapter LayersRabeeh Karimi Mahabadi, James Henderson, Sebastian RuderNeurIPS 2021 · 700 citations
  2. Improving Massively Multilingual Neural Machine Translation and Zero-Shot TranslationBiao Zhang, Philip Williams, Ivan Titov, Rico SennrichACL 2020 · 213 citations
  3. Federated Learning for Face Recognition with Gradient CorrectionYifan Niu, Weihong DengAAAI 2022 · 83 citations
  4. Making Monolingual Sentence Embeddings Multilingual using Knowledge DistillationNils Reimers, Iryna GurevychEMNLP 2020 · 54 citations
  5. Multilingual Unsupervised Neural Machine Translation with Denoising AdaptersAhmet Üstün, Alexandre Berard, Laurent Besacier, Matthias GalléEMNLP 2021 · 2 citations

Related papers

  1. Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language ModelsSajjad Ghiasvand, Yifan Yang, Zhiyu Xue, Mahnoosh Alizadeh et al.ACL Findings 2025 · 8 citations
  2. DEPT: Decoupled Embeddings for Pre-training Language ModelsAlex Iacob, Lorenzo Sani, Meghdad Kurmanji, William F. Shen et al.ICLR 2025
  3. Federated Nearest Neighbor Machine TranslationYichao Du, Zhirui Zhang, Bingzhe Wu, Lemao Liu et al.ICLR 2023 · 3 citations
  4. Improving LoRA in Privacy-preserving Federated LearningYoubang Sun, Zitao Li, Yaliang Li, Bolin DingICLR 2024 · 173 citations
  5. FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical GuaranteesJiahao Liu, Yipeng Zhou, Di Wu, Miao Hu et al.ICML 2024 · 8 citations
Dusk over the sea between two cliffs drawn in fine vertical lines