Investigating Language Preference of Multilingual RAG Systems

Multilingual Retrieval-Augmented Generation (mRAG) systems enhance language models by integrating external multilingual information to produce context-aware responses. However, mRAG systems struggle with retrieving relevant information due to linguistic variations between queries and documents, generating inconsistent responses when multilingual sources conflict. In this work, we systematically investigate language preferences in both retrieval and generation of mRAG through a series of experiments. Our analysis indicates that retrievers tend to prefer high-resource and query languages, yet this preference does not consistently improve generation performance. Moreover, we observe that generators prefer the query language or Latin scripts, leading to inconsistent outputs. To overcome these issues, we propose Dual Knowledge Multilingual RAG (DKM-RAG), a simple yet effective framework that fuses translated multilingual passages with complementary model knowledge. Empirical results demonstrate that DKM-RAG mitigates language preference in generation and enhances performance across diverse linguistic settings. Code is available at https://github.com/jeonghyunpark2002/ LanguagePreference.git † Corresponding author. ( ) . . . KO . . . 2 nd 3 rd 4 th 5 th 1 st EN EN EN ZH ES Gold Answer: Iris Mittenaere quien ganó el título de miss universe en 2016 (who won the miss universe title in 2016) Query in español #Case 1 #Case 2 LLM Retrieve Datastore Miss France 2016 was hosted by Jean-Pierre Foucault (...) Iris Mittenaere was crowned Miss Universe 2016. Not retrieved but Relevant 2 nd 3 rd 4 th 5 th 1 st EN EN EN KO ES Retrieve Datastore 𝒅𝒐𝒄𝒓𝒆𝒍 𝒅𝒐𝒄 𝒓𝒆𝒍 Generator only uses documents in the preferred languages to produce the final answer Catriona Gray Generate Incorrect Answer ( X ) 미스 프랑스 2016은 Jean-Pierre Foucault가 진행했으며 (…) Iris Mittenaere는 2016년 미스 유니버스로 선발되었습니다. Translate to 𝑳 𝒒 (ko) Gold Answer: 로버트 M. 피르식 (Robert. M. Pirsig) 누가 선과 모터사이클 관리술을 썼나요 (who wrote zen and the art of motorcycle maintenance) Query in Korean (q ∈ Q , 𝑳 𝒒 = 𝒌𝒐)