Lune

ACL Findings2023

Robustness of Multi-Source MT to Transcription Errors

Dominik Machácek, Peter Polak, Ondrej Bojar, Raj Dabre

Abstract

Automatic speech translation is sensitive to speech recognition errors, but in a multilingual scenario, the same content may be available in various languages via simultaneous interpreting, dubbing or subtitling. In this paper, we hypothesize that leveraging multiple sources will improve translation quality if the sources complement one another in terms of correct information they contain. To this end, we first show that on a 10-hour ESIC corpus, the ASR errors in the original English speech and its simultaneous interpreting into German and Czech are mutually independent. We then use two sources, English and German, in a multi-source setting for translation into Czech to establish its robustness to ASR errors. Furthermore, we observe this robustness when translating both noisy sources together in a simultaneous translation setting. Our results show that multisource neural machine translation has the potential to be useful in a real-time simultaneous translation setting, thereby motivating further investigation in this area.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on2

  1. Speech Translation and the End-to-End Promise: Taking Stock of Where We AreMatthias Sperber, Matthias PaulikACL 2020 · 7 citations
  2. VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and InterpretationChanghan Wang, Morgane Rivière, Ann Lee, Anne Wu et al.ACL 2021

Related papers

  1. Visual Cues and Error Correction for Translation RobustnessZhenhao Li, Marek Rei, Lucia SpeciaEMNLP Findings 2021 · 5 citations
  2. Improving Multilingual Neural Machine Translation with Auxiliary Source LanguagesWeijia Xu, Yuwei Yin, Shuming Ma, Dongdong Zhang et al.EMNLP Findings 2021 · 6 citations
  3. From Tens of Hours to Tens of Thousands: Scaling Back-Translation for Speech RecognitionTianduo Wang, Lu Xu, Wei Lu, Shanbo ChengEMNLP 2025 · 1 citation
  4. Multilingual Simultaneous Neural Machine TranslationPhilip Arthur, Dongwon Ryu, Gholamreza HaffariACL Findings 2021 · 3 citations
  5. Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASRJun-Kun Chen, Mingbo Ma, Renjie Zheng, Liang HuangACL Findings 2021 · 21 citations
Dusk over the sea between two cliffs drawn in fine vertical lines