Lune

EMNLP2022Top-tier venue

Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval

Houxing Ren, Linjun Shou, Ning Wu, Ming Gong, Daxin Jiang

Abstract

In monolingual dense retrieval, lots of works focus on how to distill knowledge from cross-encoder re-ranker to dual-encoder retriever and these methods achieve better performance due to the effectiveness of cross-encoder re-ranker. However, we find that the performance of the cross-encoder re-ranker is heavily influenced by the number of training samples and the quality of negative samples, which is hard to obtain in the cross-lingual setting. In this paper, we propose to use a query generator as the teacher in the cross-lingual setting, which is less dependent on enough training samples and high-quality negative samples. In addition to traditional knowledge distillation, we further propose a novel enhancement method, which uses the query generator to help the dual-encoder align queries from different languages, but does not need any additional parallel sentences. The experimental results show that our method outperforms the state-of-the-art methods on two benchmark datasets.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
  • Pre-training Cross-lingual Open Domain Question Answering with Large-scale Synthetic Supervision cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
  • Pre-training Cross-lingual Open Domain Question Answering with Large-scale Synthetic Supervision cites ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Pre-training Cross-lingual Open Domain Question Answering with Large-scale Synthetic SupervisionFan Jiang, Tom Drummond, Trevor CohnEMNLP 2024 · 1 citation
  2. Alignment with Fill-In-the-Middle for Enhancing Code GenerationHouxing Ren, Zimu Lu, Weikang Shi, Haotian Hou et al.EMNLP 2025

Builds on20

  1. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
  2. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTOmar Khattab, Matei ZahariaSIGIR 2020 · 1,246 citations
  3. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than GeneratorsKevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. ManningICLR 2020 · 541 citations
  4. Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
  5. Document Ranking with a Pretrained Sequence-to-Sequence ModelRodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, Jimmy LinEMNLP Findings 2020 · 469 citations

Related papers

  1. ADAM: Dense Retrieval Distillation with Adaptive Dark ExamplesChongyang Tao, Chang Liu, Tao Shen, Can Xu et al.ACL Findings 2024 · 2 citations
  2. Boosting Data Utilization for Multilingual Dense RetrievalChao Huang, Fengran Mo, Yufeng Chen, Changhao Guan et al.EMNLP 2025 · 2 citations
  3. PairDistill: Pairwise Relevance Distillation for Dense RetrievalChao-Wei Huang, Yun-Nung ChenEMNLP 2024 · 3 citations
  4. Distillation Enhanced Generative RetrievalYongqi Li, Zhen Zhang, Wenjie Wang, Liqiang Nie et al.ACL Findings 2024 · 6 citations
  5. Soft Prompt Decoding for Multilingual Dense RetrievalZhiqi Huang, Hansi Zeng, Hamed Zamani, James AllanSIGIR 2023 · 10 citations
Dusk over the sea between two cliffs drawn in fine vertical lines