ACL Findings2024
Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding
Jiali Zeng, Fandong Meng, Yongjing Yin, Jie Zhou
Abstract
Contemporary translation engines based on the encoder-decoder framework have made significant strides in development. However, the emergence of Large Language Models (LLMs) has disrupted their position by presenting the potential for achieving superior translation quality. To uncover the circumstances in which LLMs excel and explore how their strengths can be harnessed to enhance translation quality, we first conduct a comprehensive analysis to assess the strengths and limitations of various commercial NMT systems and MT-oriented LLMs.
Our findings indicate that neither NMT nor MToriented LLMs alone can effectively address all the translation issues, but MT-oriented LLMs show promise as a complementary solution to NMT systems. Building upon these insights, we propose Cooperative Decoding (CoDec), which treats NMT systems as a pretranslation model and MT-oriented LLMs as a supplemental solution to handle complex scenarios beyond the capability of NMT alone. Experimental results on the WMT22 test sets and a newly collected test set WebCrawl demonstrate the effectiveness and efficiency of CoDec, highlighting its potential as a robust solution for combining NMT systems with MT-oriented LLMs in the field of machine translation.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- TinyBERT: Distilling BERT for Natural Language Understanding cites MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
- TinyBERT: Distilling BERT for Natural Language Understanding cites Reducing Transformer Depth on Demand with Structured Dropout
- Teaching Large Language Models to Translate with Comparison cites Language Models are Few-Shot Learners
- A Solver-in-the-Loop Framework for Improving LLMs on Answer Set Programming for Logic Puzzle Solving cites Language Models are Few-Shot Learners
- Fast Inference from Transformers via Speculative Decoding cites Language Models are Few-Shot Learners
Cited by top-tier papers4
- Teaching Large Language Models to Translate with ComparisonJiali Zeng, Fandong Meng, Yongjing Yin, Jie ZhouAAAI 2024 · 74 citations
- A Solver-in-the-Loop Framework for Improving LLMs on Answer Set Programming for Logic Puzzle SolvingTimo Pierre Schrader, Lukas Lange, Tobias Kaminski, Simon Razniewski et al.AAAI 2026 · 4 citations
- Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine TranslationAndong Chen, Yuchen Song, Kehai Chen, Xuefeng Bai et al.ACL 2025
- G-DIG: Towards Gradient-based DIverse and hiGh-quality Instruction Data Selection for Machine TranslationXingyuan Pan, Luyang Huang, Liyan Kang, Zhicheng Liu et al.ACL 2024
Builds on17
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained TransformersWenhui Wang, Furu Wei, Li Dong, Hangbo Bao et al.NeurIPS 2020 · 2,727 citations
- TinyBERT: Distilling BERT for Natural Language UnderstandingXiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang et al.EMNLP Findings 2020 · 1,787 citations
- Fast Inference from Transformers via Speculative DecodingYaniv Leviathan, Matan Kalman, Yossi MatiasICML 2023 · 1,472 citations
- Reducing Transformer Depth on Demand with Structured DropoutAngela Fan, Edouard Grave, Armand JoulinICLR 2020 · 695 citations
Related papers
- Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine TranslationYingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li et al.ACL Findings 2025 · 5 citations
- TasTe: Teaching Large Language Models to Translate through Self-ReflectionYutong Wang, Jiali Zeng, Xuebo Liu, Fandong Meng et al.ACL 2024
- Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM TranslationZhanglin Wu, Daimeng Wei, Xiaoyu Chen, Hengchao Shang et al.ACL Findings 2025 · 4 citations
- What do Large Language Models Need for Machine Translation Evaluation?Shenbin Qian, Archchana Sindhujan, Minnie Kabra, Diptesh Kanojia et al.EMNLP 2024 · 4 citations
- M²PO: Multi-Perspective Multi-Pair Preference Optimization for Machine TranslationHao Wang, Linlong Xu, Heng Liu, Yangyang Liu et al.ACL 2026
