ACL Findings2023
Rethinking Round-Trip Translation for Machine Translation Evaluation
Abstract
Automatic evaluation methods for translation often require model training, and thus the availability of parallel corpora limits their applicability to low-resource settings. Round-trip translation is a potential workaround, which can reframe bilingual evaluation into a much simpler monolingual task. Early results from the era of statistical machine translation (SMT) raised fundamental concerns about the utility of this approach, based on poor correlation with human translation quality judgments. In this paper, we revisit this technique with modern neural translation (NMT) and show that round-trip translation does allow for accurate automatic evaluation without the need for reference translations. These opposite findings can be explained through the copy mechanism in SMT that is absent in NMT. We demonstrate that round-trip translation benefits multiple machine translation evaluation tasks: i) predicting forward translation scores; ii) improving the performance of a quality estimation model; and iii) identifying adversarial competitors in shared tasks via cross-system verification. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Co-Eval: Augmenting LLM-based Evaluation with Machine Metrics cites BERTScore: Evaluating Text Generation with BERT
- BLEURT: Learning Robust Metrics for Text Generation cites BERTScore: Evaluating Text Generation with BERT
- On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation cites BERTScore: Evaluating Text Generation with BERT
Cited by top-tier papers2
- Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example GenerationQianli Wang, Van Bach Nguyen, Yihong Liu, Fedor Splitt et al.ACL 2026
- Co-Eval: Augmenting LLM-based Evaluation with Machine MetricsLing-I Wu, Weijie Wu, Minyu Chen, Jianxin Xue et al.EMNLP 2025
Builds on11
- BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
- Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 3,729 citations
- On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation EvaluationWei Zhao, Goran Glavas, Maxime Peyrard, Yang Gao et al.ACL 2020 · 53 citations
- MetaMT, a Meta Learning Method Leveraging Multiple Domain Data for Low Resource Machine TranslationRumeng Li, Xun Wang, Hong YuAAAI 2020 · 42 citations
- BLEURT: Learning Robust Metrics for Text GenerationThibault Sellam, Dipanjan Das, Ankur P. ParikhACL 2020 · 40 citations
Related papers
- Automatic Machine Translation Evaluation in Many Languages via Zero-Shot ParaphrasingBrian Thompson, Matt PostEMNLP 2020 · 7 citations
- Multi-Hypothesis Machine Translation EvaluationMarina Fomicheva, Lucia Specia, Francisco GuzmánACL 2020 · 13 citations
- COMET: A Neural Framework for MT EvaluationRicardo Rei, Craig Stewart, Ana C. Farinha, Alon LavieEMNLP 2020 · 6 citations
- Lost in the Source Language: How Large Language Models Evaluate the Quality of Machine TranslationXu Huang, Zhirui Zhang, Xiang Geng, Yichao Du et al.ACL Findings 2024 · 9 citations
- Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?Pei Zhang, Baosong Yang, Haoran Wei, Dayiheng Liu et al.EMNLP 2022 · 1 citation
