ACL Findings2025
LLMs Can Achieve High-quality Simultaneous Machine Translation as Efficiently as Offline
Biao Fu, Minpeng Liao, Kai Fan, Chengxi Li, Liang Zhang, Yidong Chen, Xiaodong Shi
Abstract
When the complete source sentence is provided, Large Language Models (LLMs) perform excellently in offline machine translation even with a simple prompt "Translate the following sentence from [src lang] into [tgt lang]:". However, in many real scenarios, the source tokens arrive in a streaming manner and simultaneous machine translation (SiMT) is required, then the efficiency and performance of decoder-only LLMs are significantly limited by their auto-regressive nature. To enable LLMs to achieve high-quality SiMT as efficiently as offline translation, we propose a novel paradigm that includes constructing supervised fine-tuning (SFT) data for SiMT, along with new training and inference strategies. To replicate the token input/output stream in SiMT, the source and target tokens are rearranged into an interleaved sequence, separated by special tokens according to varying latency requirements. This enables powerful LLMs to learn read and write operations adaptively, based on varying latency prompts, while still maintaining efficient auto-regressive decoding. Experimental results show that, even with limited SFT data, our approach achieves stateof-the-art performance across various SiMT benchmarks, and preserves the original abilities of offline translation. Moreover, our approach generalizes well to document-level SiMT setting without requiring specific fine-tuning, even beyond the offline translation model 1 .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation cites BLEURT: Learning Robust Metrics for Text Generation
- Efficient and Adaptive Simultaneous Speech Translation with Fully Unidirectional Architecture cites Cross Attention Augmented Transducer Networks for Simultaneous Translation
Cited by top-tier papers3
- COSM: A Cooperative Scheduling Framework for Concurrent PIM and CPU Execution on Mobile DevicesYilong Zhao, Fangxin Liu, Onur Mutlu, Mingyu Gao et al.ISCA 2026 · 1 citation
- Efficient and Adaptive Simultaneous Speech Translation with Fully Unidirectional ArchitectureBiao Fu, Donglei Yu, Minpeng Liao, Chengxi Li et al.AAAI 2026 · 1 citation
- ProactiveLLM: Learning Active Interaction for Streaming Large Language ModelsJunlong Tong, Yao Zhang, Anhao Zhao, Yingqi Fan et al.ICML 2026
Builds on21
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine TranslationHaoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan et al.ICML 2024 · 447 citations
- Learning Adaptive Segmentation Policy for Simultaneous TranslationRuiqing Zhang, Chuanqiang Zhang, Zhongjun He, Hua Wu et al.EMNLP 2020 · 41 citations
- BLEURT: Learning Robust Metrics for Text GenerationThibault Sellam, Dipanjan Das, Ankur P. ParikhACL 2020 · 40 citations
- Cross Attention Augmented Transducer Networks for Simultaneous TranslationDan Liu, Mengge Du, Xiaoxi Li, Ya Li et al.EMNLP 2021 · 28 citations
Related papers
- Simul-LLM: A Framework for Exploring High-Quality Simultaneous Translation with Large Language ModelsVictor Agostinelli, Max Wild, Matthew Raffel, Kazi Ahmed Asif Fuad et al.ACL 2024 · 4 citations
- Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language PairYusuke Sakai, Mana Makinae, Hidetaka Kamigaito, Taro WatanabeEMNLP 2024 · 3 citations
- SimulS2S-LLM: Unlocking Simultaneous Inference of Speech LLMs for Speech-to-Speech TranslationKeqi Deng, Wenxi Chen, Xie Chen, Philip C. WoodlandACL 2025
- Self-Modifying State Modeling for Simultaneous Machine TranslationDonglei Yu, Xiaomian Kang, Yuchen Liu, Yu Zhou et al.ACL 2024
- Learning Optimal Policy for Simultaneous Machine Translation via Binary SearchShoutao Guo, Shaolei Zhang, Yang FengACL 2023 · 9 citations
