EMNLP Findings2021
Counter-Interference Adapter for Multilingual Machine Translation
Yaoming Zhu, Jiangtao Feng, Chengqi Zhao, Mingxuan Wang, Lei Li
Abstract
Developing a unified multilingual model has long been a pursuit for machine translation. However, existing approaches suffer from performance degradation -a single multilingual model is inferior to separately trained bilingual ones on rich-resource languages. We conjecture that such a phenomenon is due to interference caused by joint training with multiple languages. To accommodate the issue, we propose CIAT, an adapted Transformer model with a small parameter overhead for multilingual machine translation. We evaluate CIAT on multiple benchmark datasets, including IWSLT, OPUS-100, and WMT. Experiments show that CIAT consistently outperforms strong multilingual baselines on 64 of total 66 language directions, 42 of which see above 0.5 BLEU improvement. Our code is available at https://github.com/ Yaoming95/CIAT .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
23 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers23
- OWL: A Large Language Model for IT OperationsHongcheng Guo, Jian Yang, Jiaheng Liu, Liqun Yang et al.ICLR 2024 · 64 citations
- Spectral Adapter: Fine-Tuning in Spectral SpaceFangzhao Zhang, Mert PilanciNeurIPS 2024 · 34 citations
- Hyperbolic Fine-Tuning for Large Language ModelsMenglin Yang, Ram Samarth B. B., Aosong Feng, Bo Xiong et al.NeurIPS 2025 · 31 citations
- AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear MappingHaonan Dong, Wenhao Zhu, Guojie Song, Liang WangNeurIPS 2025 · 31 citations
- Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud AnalysisXin Zhou, Dingkang Liang, Wei Xu, Xingkui Zhu et al.CVPR 2024 · 25 citations
Builds on3
- Improving Massively Multilingual Neural Machine Translation and Zero-Shot TranslationBiao Zhang, Philip Williams, Ivan Titov, Rico SennrichACL 2020 · 213 citations
- Share or Not? Learning to Schedule Language-Specific Capacity for Multilingual TranslationBiao Zhang, Ankur Bapna, Rico Sennrich, Orhan FiratICLR 2021 · 97 citations
- Pre-training Multilingual Neural Machine Translation by Leveraging Alignment InformationZehui Lin, Xiao Pan, Mingxuan Wang, Xipeng Qiu et al.EMNLP 2020 · 82 citations
Related papers
- Learning Language Specific Sub-network for Multilingual Machine TranslationZehui Lin, Liwei Wu, Mingxuan Wang, Lei LiACL 2021
- switch-GLAT: Multilingual Parallel Machine Translation Via Code-Switch DecoderZhenqiao Song, Hao Zhou, Lihua Qian, Jingjing Xu et al.ICLR 2022 · 13 citations
- Contrastive Learning for Many-to-many Multilingual Neural Machine TranslationXiao Pan, Mingxuan Wang, Liwei Wu, Lei LiACL 2021
- Lego-MT: Learning Detachable Models for Massively Multilingual Machine TranslationFei Yuan, Yinquan Lu, Wenhao Zhu, Lingpeng Kong et al.ACL Findings 2023 · 4 citations
- Knowledge Distillation for Multilingual Unsupervised Neural Machine TranslationHaipeng Sun, Rui Wang, Kehai Chen, Masao Utiyama et al.ACL 2020 · 37 citations
