Few-Shot Conversational Dense Retrieval
Shi Yu, Zhenghao Liu, Chenyan Xiong, Tao Feng, Zhiyuan Liu
Abstract
Dense retrieval (DR) has the potential to resolve the query understanding challenge in conversational search by matching in the learned embedding space. However, this adaptation is challenging due to DR models' extra needs for supervision signals and the long-tail nature of conversational search. In this paper, we present a Conversational Dense Retrieval system, ConvDR, that learns contextualized embeddings for multi-turn conversational queries and retrieves documents solely using embedding dot products. In addition, we grant ConvDR few-shot ability using a teacher-student framework, where we employ an ad hoc dense retriever as the teacher, inherit its document encodings, and learn a student query encoder to mimic the teacher embeddings on oracle reformulated queries. Our experiments on TREC CAsT and OR-QuAC demonstrate ConvDR's effectiveness in both few-shot and fully-supervised settings. It outperforms previous systems that operate in the sparse word space, matches the retrieval accuracy of oracle query reformulations, and is also more efficient thanks to its simplicity. Our analyses reveal that the advantages of ConvDR come from its ability to capture informative context while ignoring the unrelated context in previous conversation rounds. This makes ConvDR more effective as conversations evolve while previous systems may get confused by the increased noise from previous turns. Our code is publicly available at https://github.com/thunlp/ConvDR.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
36 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Learning Denoised and Interpretable Session Representation for Conversational Search cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
- Learning Denoised and Interpretable Session Representation for Conversational Search cites CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning
- Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
- CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
- Dialog Inpainting: Turning Documents into Dialogs cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
- Fine-Grained Distillation for Long Document Retrieval cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Cited by top-tier papers36
- Dialog Inpainting: Turning Documents into DialogsZhuyun Dai, Arun Tejasvi Chaganty, Vincent Y. Zhao, Aida Amini et al.ICML 2022 · 77 citations
- Fine-Grained Distillation for Long Document RetrievalYucheng Zhou, Tao Shen, Xiubo Geng, Chongyang Tao et al.AAAI 2024 · 44 citations
- MultiDoc2Dial: Modeling Dialogues Grounded in Multiple DocumentsSong Feng, Siva Sankalp Patel, Hui Wan, Sachindra JoshiEMNLP 2021 · 42 citations
- Learning Denoised and Interpretable Session Representation for Conversational SearchKelong Mao, Hongjin Qian, Fengran Mo, Zhicheng Dou et al.WWW 2023 · 38 citations
- CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement LearningZeqiu Wu, Yi Luan, Hannah Rashkin, David Reitter et al.EMNLP 2022 · 37 citations
Builds on10
- The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the HypersphereTongzhou Wang, Phillip IsolaICML 2020 · 2,360 citations
- Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
- ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTOmar Khattab, Matei ZahariaSIGIR 2020 · 1,246 citations
- Generating Clarifying Questions for Information RetrievalHamed Zamani, Susan T. Dumais, Nick Craswell, Paul N. Bennett et al.WWW 2020 · 238 citations
Related papers
- History-Aware Conversational Dense RetrievalFengran Mo, Chen Qu, Kelong Mao, Tianyu Zhu et al.ACL Findings 2024 · 7 citations
- Curriculum Contrastive Context Denoising for Few-shot Conversational Dense RetrievalKelong Mao, Zhicheng Dou, Hongjin QianSIGIR 2022 · 40 citations
- ConvTrans: Transforming Web Search Sessions for Conversational Dense RetrievalKelong Mao, Zhicheng Dou, Hongjin Qian, Fengran Mo et al.EMNLP 2022 · 12 citations
- Open-Retrieval Conversational Question AnsweringChen Qu, Liu Yang, Cen Chen, Minghui Qiu et al.SIGIR 2020 · 84 citations
- Explicit Query Rewriting for Conversational Dense RetrievalHongjin Qian, Zhicheng DouEMNLP 2022 · 14 citations
