Lune

EMNLP Findings2021

Stream-level Latency Evaluation for Simultaneous Machine Translation

Javier Iranzo-Sánchez, Jorge Civera Saiz, Alfons Juan

Abstract

Simultaneous machine translation has recently gained traction thanks to significant quality improvements and the advent of streaming applications. Simultaneous translation systems need to find a trade-off between translation quality and response time, and with this purpose multiple latency measures have been proposed. However, latency evaluations for simultaneous translation are estimated at the sentence level, not taking into account the sequential nature of a streaming scenario. Indeed, these sentence-level latency measures are not well suited for continuous stream translation, resulting in figures that are not coherent with the simultaneous translation policy of the system being assessed. This work proposes a stream-level adaptation of the current latency measures based on a re-segmentation approach applied to the output translation, that is successfully evaluated on streaming conditions for a reference IWSLT task.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • From Simultaneous to Streaming Machine Translation by Leveraging Streaming History cites Direct Segmentation Models for Streaming Speech Translation
  • StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection cites From Simultaneous to Streaming Machine Translation by Leveraging Streaming History
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Learning Optimal Policy for Simultaneous Machine Translation via Binary SearchShoutao Guo, Shaolei Zhang, Yang FengACL 2023 · 9 citations
  2. SimQA: Detecting Simultaneous MT Errors through Word-by-Word Question AnsweringHyoJung Han, Marine Carpuat, Jordan L. Boyd-GraberEMNLP 2022 · 3 citations
  3. From Simultaneous to Streaming Machine Translation by Leveraging Streaming HistoryJavier Iranzo-Sánchez, Jorge Civera, Alfons Juan-CíscarACL 2022
  4. StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History SelectionSara Papi, Marco Gaido, Matteo Negri, Luisa BentivogliACL 2024

Builds on1

  1. Direct Segmentation Models for Streaming Speech TranslationJavier Iranzo-Sánchez, Adrià Giménez-Pastor, Joan Albert Silvestre-Cerdà, Pau Baquero-Arnal et al.EMNLP 2020 · 24 citations

Related papers

  1. Learning Adaptive Segmentation Policy for Simultaneous TranslationRuiqing Zhang, Chuanqiang Zhang, Zhongjun He, Hua Wu et al.EMNLP 2020 · 41 citations
  2. Going Beyond Your Expectations in Latency Metrics for Simultaneous Speech TranslationJorge Iranzo-Sánchez, Javier Iranzo-Sánchez, Adrià Giménez, Jorge CiveraACL Findings 2025
  3. Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine TranslationShoutao Guo, Shaolei Zhang, Yang FengEMNLP Findings 2022 · 10 citations
  4. Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive TrainingRenjie Zheng, Mingbo Ma, Baigong Zheng, Kaibo Liu et al.EMNLP Findings 2020 · 11 citations
  5. SimulS2S-LLM: Unlocking Simultaneous Inference of Speech LLMs for Speech-to-Speech TranslationKeqi Deng, Wenxi Chen, Xie Chen, Philip C. WoodlandACL 2025
Dusk over the sea between two cliffs drawn in fine vertical lines