Lune

ACL Findings2022

End-to-End Speech Translation for Code Switched Speech

Orion Weller, Matthias Sperber, Telmo Pires, Hendra Setiawan, Christian Gollan, Dominic Telaar, Matthias Paulik

Abstract

Code switching (CS) refers to the phenomenon of interchangeably using words and phrases from different languages. CS can pose significant accuracy challenges to NLP, due to the often monolingual nature of the underlying systems. In this work, we focus on CS in the context of English/Spanish conversations for the task of speech translation (ST), generating and evaluating both transcript and translation. To evaluate model performance on this task, we create a novel ST corpus derived from existing public data sets. 1 We explore various ST architectures across two dimensions: cascaded (transcribe then translate) vs end-toend (jointly transcribe and translate) and unidirectional (source → target) vs bidirectional (source ↔ target). We show that our ST architectures, and especially our bidirectional end-to-end architecture, perform well on CS speech, even when no CS training data is used.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Beyond Monolingual Assumptions: A Survey on Code-Switched NLP in the Era of Large Language Models across ModalitiesRajvee Sheth, Samridhi Raj Sinha, Mahavir Patil, Himanshu Beniwal et al.ACL 2026 · 2 citations

Builds on3

  1. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech RepresentationsAlexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, Michael AuliNeurIPS 2020 · 9,451 citations
  2. CSP: Code-Switching Pre-training for Neural Machine TranslationZhen Yang, Bojie Hu, Ambyera Han, Shen Huang et al.EMNLP 2020 · 62 citations
  3. Speech Translation and the End-to-End Promise: Taking Stock of Where We AreMatthias Sperber, Matthias PaulikACL 2020 · 7 citations

Related papers

  1. Code-switching Mediated Sentence-level Semantic LearningShuai Zhang, Jiangyan Yi, Zhengqi Wen, Jianhua Tao et al.AAAI 2025
  2. End-to-End Single-Channel Speaker-Turn Aware Conversational Speech TranslationJuan Pablo Zuluaga-Gomez, Zhaocheng Huang, Xing Niu, Rohit Paturi et al.EMNLP 2023 · 2 citations
  3. Automatic Identification of Code-Switching Functions in Speech TranscriptsRitu Belani, Jeffrey FlaniganACL Findings 2023 · 1 citation
  4. The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech TranslationMutian He, Philip N. GarnerEMNLP Findings 2023 · 2 citations
  5. Discourse-Driven Code-Switching: Analyzing the Role of Content and Communicative Function in Spanish-English Bilingual SpeechDebasmita Bhattacharya, Juan Junco, Divya Tadimeti, Julia HirschbergEMNLP 2025
Dusk over the sea between two cliffs drawn in fine vertical lines