Lune

EMNLP Findings2023

Detrimental Contexts in Open-Domain Question Answering

Philhoon Oh, James Thorne

Abstract

For knowledge intensive NLP tasks, it has been widely accepted that accessing more information is a contributing factor to improvements in the model's end-to-end performance. However, counter-intuitively, too much context can have a negative impact on the model when evaluated on common question answering (QA) datasets. In this paper, we analyze how passages can have a detrimental effect on retrieve-then-read architectures used in question answering. Our empirical evidence indicates that the current read architecture does not fully leverage the retrieved passages and significantly degrades its performance when using the whole passages compared to utilizing subsets of them. Our findings demonstrate that model accuracy can be improved by 10% on two popular QA datasets by filtering out detrimental passages. Additionally, these outcomes are attained by utilizing existing retrieval methods without further training or data. We further highlight the challenges associated with identifying the detrimental passages. First, even with the correct context, the model can make an incorrect prediction, posing a challenge in determining which passages are most influential. Second, evaluation typically considers lexical matching, which is not robust to variations of correct answers. Despite these limitations, our experimental results underscore the pivotal role of identifying and removing these detrimental passages for the context-efficient retrieve-then-read pipeline. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Autoregressive Entity Retrieval cites Dense Passage Retrieval for Open-Domain Question Answering
  • Autoregressive Entity Retrieval cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Autoregressive Search Engines: Generating Substrings as Document Identifiers cites Dense Passage Retrieval for Open-Domain Question Answering
  • Dense Passage Retrieval for Open-Domain Question Answering cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks cites Dense Passage Retrieval for Open-Domain Question Answering
  • Distilling Knowledge from Reader to Retriever for Question Answering cites Dense Passage Retrieval for Open-Domain Question Answering
  • Distilling Knowledge from Reader to Retriever for Question Answering cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • To Generate or to Retrieve? On the Effectiveness of Artificial Contexts for Medical Open-Domain Question Answering cites Dense Passage Retrieval for Open-Domain Question Answering
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. To Generate or to Retrieve? On the Effectiveness of Artificial Contexts for Medical Open-Domain Question AnsweringGiacomo Frisoni, Alessio Cocchieri, Alex Presepi, Gianluca Moro et al.ACL 2024 · 8 citations

Builds on13

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  2. Distilling Knowledge from Reader to Retriever for Question AnsweringGautier Izacard, Edouard GraveICLR 2021 · 317 citations
  3. Autoregressive Search Engines: Generating Substrings as Document IdentifiersMichele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Scott Yih et al.NeurIPS 2022 · 242 citations
  4. Autoregressive Entity RetrievalNicola De Cao, Gautier Izacard, Sebastian Riedel, Fabio PetroniICLR 2021 · 200 citations
  5. Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis et al.EMNLP 2020 · 142 citations

Related papers

  1. The Distracting Effect: Understanding Irrelevant Passages in RAGChen Amiraz, Florin Cuconasu, Simone Filice, Zohar S. KarninACL 2025
  2. Making Retrieval-Augmented Language Models Robust to Irrelevant ContextOri Yoran, Tomer Wolfson, Ori Ram, Jonathan BerantICLR 2024 · 361 citations
  3. Optimizing Retrieval-augmented Reader Models via Token EliminationMoshe Berchansky, Peter Izsak, Avi Caciularu, Ido Dagan et al.EMNLP 2023 · 4 citations
  4. Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-kChihiro Taguchi, Seiji Maekawa, Nikita BhutaniEMNLP 2025
  5. Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader ModelsShujian Zhang, Chengyue Gong, Xingchao LiuEMNLP 2022 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines