PQR: Improving Dense Retrieval via Potential Query Modeling
Junfeng Kang, Rui Li, Qi Liu, Yanjiang Chen, Zheng Zhang, Junzhe Jiang, Heng Yu, Yu Su
Abstract
Dense retrieval has now become the mainstream paradigm in information retrieval. The core idea of dense retrieval is to align document embeddings with their corresponding query embeddings by maximizing their dot product. The current training data is quite sparse, with each document typically associated with only one or a few labeled queries. However, a single document can be retrieved by multiple different queries. Aligning a document with just one or a limited number of labeled queries results in a loss of its semantic information. In this paper, we propose a training-free Potential Query Retrieval (PQR) framework to address this issue. Specifically, we use a Gaussian mixture distribution to model all potential queries for a document, aiming to capture its comprehensive semantic information. To obtain this distribution, we introduce three sampling strategies to sample a large number of potential queries for each document and encode them into a semantic space. Using these sampled queries, we employ the Expectation-Maximization algorithm to estimate parameters of the distribution. Finally, we also propose a method to calculate similarity scores between user queries and documents under the PQR framework. Extensive experiments demonstrate the effectiveness of the proposed method. 1 1 The source code is released at https://github.com/ tojunfeng/PQR Q1: Which airlines agreed to install iPod seat connections? … Doc: Beginning in mid-2007, four major airlines, United, Continental, Delta, and Emirates, reached agreements to install iPod seat connections. The free service will allow passengers to power and charge an iPod...... Originally KLM and Air France were reported to be part of the deal with Apple, but…... The iPod line can play several audio file formats including MP3, AAC/M4A, Protected AAC, AIFF, WAV, Audible audiobook, and Apple Lossless. The iPod Photo introduced the ability to display JPEG, BMP, GIF…… … Train Stage Close Q2: What audio file formats can the iPod line play? … Distant Infer Stage
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Precise Zero-Shot Dense Retrieval without Relevance Labels cites Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
- Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Builds on13
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- SGLang: Efficient Execution of Structured Language Model ProgramsLianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun et al.NeurIPS 2024 · 1,586 citations
- Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
- Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text ModelsJianmo Ni, Gustavo Hernández Ábrego, Noah Constant, Ji Ma et al.ACL Findings 2022 · 358 citations
- Precise Zero-Shot Dense Retrieval without Relevance LabelsLuyu Gao, Xueguang Ma, Jimmy Lin, Jamie CallanACL 2023 · 211 citations
Related papers
- Distribution-Driven Dense Retrieval: Modeling Many-to-One Query-Document RelationshipJunfeng Kang, Rui Li, Qi Liu, Zhenya Huang et al.AAAI 2025 · 2 citations
- QAEncoder: Towards Aligned Representation Learning in Question Answering SystemsZhengren Wang, Qinhan Yu, Shida Wei, Zhiyu Li et al.ACL 2025 · 5 citations
- Exploring the Best Practices of Query Expansion with Large Language ModelsLe Zhang, Yihong Wu, Qian Yang, Jian-Yun NieEMNLP Findings 2024 · 12 citations
- Multi-View Document Representation Learning for Open-Domain Dense RetrievalShunyu Zhang, Yaobo Liang, Ming Gong, Daxin Jiang et al.ACL 2022 · 80 citations
- Query-as-context Pre-training for Dense Passage RetrievalXing Wu, Guangyuan Ma, Wanhui Qian, Zijia Lin et al.EMNLP 2023 · 2 citations
