Blessing of Multilinguality: A Systematic Analysis of Multilingual In-Context Learning

While multilingual large language models generally perform adequately, and sometimes even rival English performance on high-resource languages (HRLs), they often significantly underperform on low-resource languages (LRLs; Costa-jussà et al., 2022). Among several prompting strategies aiming at bridging the gap, multilingual in-context learning (ICL; Shi et al., 2023) has been particularly effective when demonstration in target languages is unavailable. However, there lacks a systematic understanding of when and why it works well.

In this work, we systematically analyze multilingual ICL, using demonstrations in HRLs to enhance cross-lingual transfer. We show that demonstrations in mixed HRLs consistently outperform English-only ones across the board, particularly for tasks written in LRLs. Surprisingly, our ablation study shows that the presence of irrelevant non-English sentences in the prompt yields measurable gains, suggesting the effectiveness of multilingual exposure itself. Our results highlight the potential of strategically leveraging multilingual resources to bridge the performance gap for underrepresented languages.

, where the split D lang ℓ train consists of M examples for any ℓ ∈ 1, . . . , L. The same applies to the test dataset D test , with each language-specific split consisting of N examples. Without further specification, we assume that the training examples at the same index are semantically equivalent across languages.

The ground-truth labels a i in quantitative LLM benchmarks (Ponti et al., 2020; Cobbe et al., 2021, inter alia) are typically language-agnostic (such as 6214