ACL Findings2023
Rethinking Translation Memory Augmented Neural Machine Translation
Hongkun Hao, Guoping Huang, Lemao Liu, Zhirui Zhang, Shuming Shi, Rui Wang
Abstract
This paper rethinks translation memory augmented neural machine translation (TMaugmented NMT) from two perspectives, i.e., a probabilistic view of retrieval and the variancebias decomposition principle. The finding demonstrates that TM-augmented NMT is good at the ability of fitting data (i.e., lower bias) but is more sensitive to the fluctuations in the training data (i.e., higher variance), which provides an explanation to a recently reported contradictory phenomenon on the same translation task: TM-augmented NMT substantially advances vanilla NMT under the high-resource scenario whereas it fails under the low-resource scenario. Then we propose a simple yet effective TM-augmented NMT model to promote the variance and address the contradictory phenomenon. Extensive experiments show that the proposed TMaugmented NMT achieves consistent gains over both conventional NMT and existing TM-augmented NMT under two variancepreferable (low-resource and plug-and-play) scenarios as well as the high-resource scenario.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach cites Boosting Neural Machine Translation with Similar Translations
- Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach cites Nearest Neighbor Machine Translation
- Efficient Cluster-Based k-Nearest-Neighbor Machine Translation cites Nearest Neighbor Machine Translation
- Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layer cites Efficient Cluster-Based k-Nearest-Neighbor Machine Translation
- Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layer cites Nearest Neighbor Machine Translation
- IMTLab: An Open-Source Platform for Building, Evaluating, and Diagnosing Interactive Machine Translation Systems cites Nearest Neighbor Machine Translation
- Nearest Neighbor Machine Translation cites Boosting Neural Machine Translation with Similar Translations
- Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data cites Nearest Neighbor Machine Translation
Cited by top-tier papers4
- Understanding Retrieval Robustness for Retrieval-augmented Image CaptioningWenyan Li, Jiaang Li, Rita Ramos, Raphael Tang et al.ACL 2024 · 5 citations
- Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection LayerRuize Gao, Zhirui Zhang, Yichao Du, Lemao Liu et al.EMNLP 2023 · 3 citations
- Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration ApproachSiqi Li, Danni Liu, Jan NiehuesEMNLP 2024 · 1 citation
- IMTLab: An Open-Source Platform for Building, Evaluating, and Diagnosing Interactive Machine Translation SystemsXu Huang, Zhirui Zhang, Ruize Gao, Yichao Du et al.EMNLP 2023
Builds on16
- Nearest Neighbor Machine TranslationUrvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer et al.ICLR 2021 · 323 citations
- Rethinking Bias-Variance Trade-off for Generalization of Neural NetworksZitong Yang, Yaodong Yu, Chong You, Jacob Steinhardt et al.ICML 2020 · 219 citations
- Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training DataShuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu et al.ACL 2022 · 115 citations
- Boosting Neural Machine Translation with Similar TranslationsJitao Xu, Josep Maria Crego, Jean SenellartACL 2020 · 59 citations
- Efficient Cluster-Based k-Nearest-Neighbor Machine TranslationDexin Wang, Kai Fan, Boxing Chen, Deyi XiongACL 2022 · 35 citations
Related papers
- Neural Machine Translation with Monolingual Translation MemoryDeng Cai, Yan Wang, Huayang Li, Wai Lam et al.ACL 2021
- Neural Machine Translation with Contrastive Translation MemoriesXin Cheng, Shen Gao, Lemao Liu, Dongyan Zhao et al.EMNLP 2022 · 10 citations
- Prompting Neural Machine Translation with Translation MemoriesAbudurexiti Reheman, Tao Zhou, Yingfeng Luo, Di Yang et al.AAAI 2023 · 11 citations
- Paraphrasing as Zero-shot Translation with Feature-guided Diversity EnhancementZiyue Yan, Hongying Zan, Xinglin Lyu, Hongfei XuACL 2026
- Deterministic Reversible Data Augmentation for Neural Machine TranslationJiashu Yao, Heyan Huang, Zeming Liu, Yuhang GuoACL Findings 2024 · 1 citation
