EMNLP Findings2021
Dense Hierarchical Retrieval for Open-domain Question Answering
Ye Liu, Kazuma Hashimoto, Yingbo Zhou, Semih Yavuz, Caiming Xiong, Philip S. Yu
Abstract
Dense neural text retrieval has achieved promising results on open-domain Question Answering (QA), where latent representations of questions and passages are exploited for maximum inner product search in the retrieval process. However, current dense retrievers require splitting documents into short passages that usually contain local, partial and sometimes biased context, and highly depend on the splitting process. As a consequence, it may yield inaccurate and misleading hidden representations, thus deteriorating the final retrieval result. In this work, we propose Dense Hierarchical Retrieval (DHR), a hierarchical framework which can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic semantics specific to each passage. Specifically, a document-level retriever first identifies relevant documents, among which relevant passages are then retrieved by a passage-level retriever. The ranking of the retrieved passages will be further calibrated by examining the document-level relevance. In addition, hierarchical title structure and two negative sampling strategies (i.e., In-Doc and In-Sec negatives) are investigated. We apply DHR to large-scale open-domain QA datasets. DHR significantly outperforms the original dense passage retriever, and helps an end-to-end QA system outperform the strong baselines on multiple open-domain QA benchmarks.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
10 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval cites Distilling Knowledge from Reader to Retriever for Question Answering
- RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval cites Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
- Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Beyond Chunking: Discourse-Aware Hierarchical Retrieval for Long Document Question Answering cites RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
- Beyond Chunking: Discourse-Aware Hierarchical Retrieval for Long Document Question Answering cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Distilling Knowledge from Reader to Retriever for Question Answering cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Distilling Knowledge from Reader to Retriever for Question Answering cites Pre-training Tasks for Embedding-based Large-scale Retrieval
- Modeling Multi-hop Question Answering as Single Sequence Prediction cites Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
- Modeling Multi-hop Question Answering as Single Sequence Prediction cites Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
- DTCRS: Dynamic Tree Construction for Recursive Summarization cites Distilling Knowledge from Reader to Retriever for Question Answering
Cited by top-tier papers10
- RAPTOR: Recursive Abstractive Processing for Tree-Organized RetrievalParth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna et al.ICLR 2024 · 460 citations
- QASA: Advanced Question Answering on Scientific ArticlesYoonjoo Lee, Kyungjae Lee, Sunghyun Park, Dasol Hwang et al.ICML 2023 · 76 citations
- Modeling Multi-hop Question Answering as Single Sequence PredictionSemih Yavuz, Kazuma Hashimoto, Yingbo Zhou, Nitish Shirish Keskar et al.ACL 2022 · 35 citations
- Beyond Chunking: Discourse-Aware Hierarchical Retrieval for Long Document Question AnsweringHuiyao Chen, Yi Yang, Yinghui Li, Meishan Zhang et al.ACL 2026 · 6 citations
- DTCRS: Dynamic Tree Construction for Recursive SummarizationGuanran Luo, Zhongquan Jian, Wentao Qiu, Meihong Wang et al.ACL 2025 · 3 citations
Builds on9
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Pre-training Tasks for Embedding-based Large-scale RetrievalWei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang et al.ICLR 2020 · 325 citations
- Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question AnsweringAkari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher et al.ICLR 2020 · 322 citations
- Distilling Knowledge from Reader to Retriever for Question AnsweringGautier Izacard, Edouard GraveICLR 2021 · 317 citations
- Answering Complex Open-Domain Questions with Multi-Hop Dense RetrievalWenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du et al.ICLR 2021 · 232 citations
Related papers
- Hybrid Hierarchical Retrieval for Open-Domain Question AnsweringManoj Ghuhan Arivazhagan, Lan Liu, Peng Qi, Xinchi Chen et al.ACL Findings 2023 · 15 citations
- Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis et al.EMNLP 2020 · 142 citations
- Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question AnsweringJiawei Zhou, Xiaoguang Li, Lifeng Shang, Lan Luo et al.ACL 2022 · 23 citations
- Structure-Aware Quantized Retrieval for Long-Document Question AnsweringHui Huang, Julien Velcin, Yacine KessaciACL Findings 2026
- You Only Need One Model for Open-domain Question AnsweringHaejun Lee, Akhil Kedia, Jongwon Lee, Ashwin Paranjape et al.EMNLP 2022
