EMNLP Findings2020
Multi-Agent Mutual Learning at Sentence-Level and Token-Level for Neural Machine Translation
Abstract
Mutual learning, where multiple agents learn collaboratively and teach one another, has been shown to be an effective way to distill knowledge for image classification tasks. In this paper, we extend mutual learning to the machine translation task and operate at both the sentence-level and the token-level. Firstly, we co-train multiple agents by using the same parallel corpora. After convergence, each agent selects and learns its poorly predicted tokens from other agents. The poorly predicted tokens are determined by the acceptance-rejection sampling algorithm. Our experiments show that sequential mutual learning at the sentence-level and the token-level improves the results cumulatively. Absolute improvements compared to strong baselines are obtained on various translation tasks. On the IWSLT'14 German-English task, we get a new state-of-the-art BLEU score of 37.0. We also report a competitive result, 29.9 BLEU score, on the WMT'14 English-German task.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers1
Related papers
- Multiscale Collaborative Deep Models for Neural Machine TranslationXiangpeng Wei, Heng Yu, Yue Hu, Yue Zhang et al.ACL 2020 · 27 citations
- Dual-teacher Knowledge Distillation for Low-frequency Word TranslationYifan Guo, Hongying Zan, Hongfei XuEMNLP Findings 2024 · 1 citation
- Jointly Masked Sequence-to-Sequence Model for Non-Autoregressive Neural Machine TranslationJunliang Guo, Linli Xu, Enhong ChenACL 2020 · 56 citations
- Transductive Ensemble Learning for Neural Machine TranslationYiren Wang, Lijun Wu, Yingce Xia, Tao Qin et al.AAAI 2020 · 29 citations
- Data Diversification: A Simple Strategy For Neural Machine TranslationXuan-Phi Nguyen, Shafiq R. Joty, Kui Wu, Ai Ti AwNeurIPS 2020 · 75 citations
