Query in Your Tongue: Reinforce Large Language Models with Retrievers for Cross-lingual Search Generative Experience

Multi-stage information retrieval (IR) has become a widely-adopted paradigm in search. While Large Language Models (LLMs) have been extensively evaluated as second-stage reranking models for monolingual IR, a systematic large-scale comparison is still lacking for cross-lingual IR (CLIR). Moreover, while prior work shows that LLM-based rerankers improve CLIR performance, their evaluation setup relies on lexical retrieval with machine translation (MT) for the first stage. This is not only prohibitively expensive but also prone to error propagation across stages. Our evaluation on passage-level and document-level CLIR reveals that further gains can be achieved with multilingual bi-encoders as first-stage retrievers and that the benefits of translation diminishes with stronger reranking models. We further show that pairwise rerankers based on instructiontuned LLMs perform competitively with listwise rerankers. To the best of our knowledge, we are the first to study the interaction between retrievers and rerankers in two-stage CLIR with LLMs. Our findings reveal that, without MT, current state-of-the-art rerankers fall severely short when directly applied in CLIR. * Equal contribution. unavailable or unreliable for many low-resource languages (Haddow et al., 2022) , adversely affecting cross-lingual retrieval when translations contain errors (Litschko et al., 2022a; Guo et al., 2024) . Recent work leveraging large language models (LLMs) in information retrieval has demonstrated promising gains over baseline systems (Ma et al., 2024 (Ma et al., , 2023)) , highlighting their capability in ranking tasks. While prior work focuses on monolingual retrieval and reranking, cross-lingual LLMbased retrieval has been understudied. To the best of our knowledge, the only prior works on LLMbased CLIR rely on MT to bridge the language gap in the retrieval stage (Adeyemi et al., 2024a)