Lune

ACL Findings2023

Code-Switched Text Synthesis in Unseen Language Pairs

I-Hung Hsu, Avik Ray, Shubham Garg, Nanyun Peng, Jing Huang

Abstract

Existing efforts on text synthesis for codeswitching mostly require training on codeswitched texts in the target language pairs, limiting the deployment of the models to cases lacking code-switched data.In this work, we study the problem of synthesizing codeswitched texts for language pairs absent from the training data.We introduce GLOSS, a model built on top of a pre-trained multilingual machine translation model (PMMTM) with an additional code-switching module.This module, either an adapter or extra prefixes, learns code-switching patterns from codeswitched data during training, while the primary component of GLOSS, i.e., the PMMTM, is frozen.The design of only adjusting the code-switching module prevents our model from overfitting to the constrained training data for code-switching.Hence, GLOSS exhibits the ability to generalize and synthesize codeswitched texts across a broader spectrum of language pairs.Additionally, we develop a self-training algorithm on target language pairs further to enhance the reliability of GLOSS.Automatic evaluations on four language pairs show that GLOSS achieves at least 55% relative BLEU and METEOR scores improvements compared to strong baselines.Human evaluations on two language pairs further validate the success of GLOSS.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages cites A Semi-supervised Approach to Generate the Code-Mixed Text using Pre-trained Encoder and Transfer Learning
  • CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages cites The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges
  • Beyond Monolingual Assumptions: A Survey on Code-Switched NLP in the Era of Large Language Models across Modalities cites CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages
  • Beyond Monolingual Assumptions: A Survey on Code-Switched NLP in the Era of Large Language Models across Modalities cites The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Beyond Monolingual Assumptions: A Survey on Code-Switched NLP in the Era of Large Language Models across ModalitiesRajvee Sheth, Samridhi Raj Sinha, Mahavir Patil, Himanshu Beniwal et al.ACL 2026 · 2 citations
  2. CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 LanguagesYilun Yang, Yekun ChaiEMNLP 2025 · 1 citation

Builds on11

  1. Towards a Unified View of Parameter-Efficient Transfer LearningJunxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick et al.ICLR 2022 · 1,182 citations
  2. MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse LanguagesJack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie et al.ACL 2023 · 88 citations
  3. The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and ChallengesGenta Indra Winata, Alham Fikri Aji, Zheng Xin Yong, Thamar SolorioACL Findings 2023 · 46 citations
  4. A Semi-supervised Approach to Generate the Code-Mixed Text using Pre-trained Encoder and Transfer LearningDeepak Gupta, Asif Ekbal, Pushpak BhattacharyyaEMNLP Findings 2020 · 44 citations
  5. Measuring and Mitigating Name Biases in Neural Machine TranslationJun Wang, Benjamin I. P. Rubinstein, Trevor CohnACL 2022 · 31 citations

Related papers

  1. From Machine Translation to Code-Switching: Generating High-Quality Code-Switched TextIshan Tarunesh, Syamantak Kumar, Preethi JyothiACL 2021
  2. Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-SwitchingParul Chopra, Sai Krishna Rallabandi, Alan W. Black, Khyathi Raghavi ChanduEMNLP Findings 2021 · 2 citations
  3. Alternating Language Modeling for Cross-Lingual Pre-TrainingJian Yang, Shuming Ma, Dongdong Zhang, Shuangzhi Wu et al.AAAI 2020 · 94 citations
  4. Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-TrainingZhijun Wang, Jiahuan Li, Hao Zhou, Rongxiang Weng et al.ACL Findings 2025 · 3 citations
  5. switch-GLAT: Multilingual Parallel Machine Translation Via Code-Switch DecoderZhenqiao Song, Hao Zhou, Lihua Qian, Jingjing Xu et al.ICLR 2022 · 13 citations
Dusk over the sea between two cliffs drawn in fine vertical lines