Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval
Houxing Ren, Linjun Shou, Ning Wu, Ming Gong, Daxin Jiang
Abstract
In monolingual dense retrieval, lots of works focus on how to distill knowledge from cross-encoder re-ranker to dual-encoder retriever and these methods achieve better performance due to the effectiveness of cross-encoder re-ranker. However, we find that the performance of the cross-encoder re-ranker is heavily influenced by the number of training samples and the quality of negative samples, which is hard to obtain in the cross-lingual setting. In this paper, we propose to use a query generator as the teacher in the cross-lingual setting, which is less dependent on enough training samples and high-quality negative samples. In addition to traditional knowledge distillation, we further propose a novel enhancement method, which uses the query generator to help the dual-encoder align queries from different languages, but does not need any additional parallel sentences. The experimental results show that our method outperforms the state-of-the-art methods on two benchmark datasets.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
- Pre-training Cross-lingual Open Domain Question Answering with Large-scale Synthetic Supervision cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
- Pre-training Cross-lingual Open Domain Question Answering with Large-scale Synthetic Supervision cites ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Cited by top-tier papers2
Builds on20
- Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
- ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTOmar Khattab, Matei ZahariaSIGIR 2020 · 1,246 citations
- ELECTRA: Pre-training Text Encoders as Discriminators Rather Than GeneratorsKevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. ManningICLR 2020 · 541 citations
- Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
- Document Ranking with a Pretrained Sequence-to-Sequence ModelRodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, Jimmy LinEMNLP Findings 2020 · 469 citations
Related papers
- ADAM: Dense Retrieval Distillation with Adaptive Dark ExamplesChongyang Tao, Chang Liu, Tao Shen, Can Xu et al.ACL Findings 2024 · 2 citations
- Boosting Data Utilization for Multilingual Dense RetrievalChao Huang, Fengran Mo, Yufeng Chen, Changhao Guan et al.EMNLP 2025 · 2 citations
- PairDistill: Pairwise Relevance Distillation for Dense RetrievalChao-Wei Huang, Yun-Nung ChenEMNLP 2024 · 3 citations
- Distillation Enhanced Generative RetrievalYongqi Li, Zhen Zhang, Wenjie Wang, Liqiang Nie et al.ACL Findings 2024 · 6 citations
- Soft Prompt Decoding for Multilingual Dense RetrievalZhiqi Huang, Hansi Zeng, Hamed Zamani, James AllanSIGIR 2023 · 10 citations
