Lune

EMNLP Findings2023

Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering

Sukmin Cho, Jeongyeon Seo, Soyeong Jeong, Jong C. Park

Abstract

Large language models (LLMs) enable zeroshot approaches in open-domain question answering (ODQA), yet with limited advancements as the reader is compared to the retriever. This study aims at the feasibility of a zero-shot reader that addresses the challenges of computational cost and the need for labeled data. We find that LLMs are distracted due to irrelevant documents in the retrieved set and the overconfidence of the generated answers when they are exploited as zero-shot readers. To tackle these problems, we mitigate the impact of such documents via Distraction-aware Answer Selection (DAS) with a negation-based instruction and score adjustment for proper answer selection. Experimental results show that our approach successfully handles distraction across diverse scenarios, enhancing the performance of zeroshot readers. Furthermore, unlike supervised readers struggling with unseen data, zero-shot readers demonstrate outstanding transferability without any training.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Large Language Models Can Be Easily Distracted by Irrelevant Context cites Multitask Prompted Training Enables Zero-Shot Task Generalization
  • Large Language Models Can Be Easily Distracted by Irrelevant Context cites Finetuned Language Models are Zero-Shot Learners
  • Generate rather than Retrieve: Large Language Models are Strong Context Generators cites Dense Passage Retrieval for Open-Domain Question Answering
  • Generate rather than Retrieve: Large Language Models are Strong Context Generators cites Finetuned Language Models are Zero-Shot Learners
  • Multitask Prompted Training Enables Zero-Shot Task Generalization cites Finetuned Language Models are Zero-Shot Learners
  • Finetuned Language Models are Zero-Shot Learners cites Multitask Prompted Training Enables Zero-Shot Task Generalization
  • Database-Augmented Query Representation for Information Retrieval cites Dense Passage Retrieval for Open-Domain Question Answering
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Database-Augmented Query Representation for Information RetrievalSoyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang et al.EMNLP 2025

Builds on10

  1. Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
  2. Multitask Prompted Training Enables Zero-Shot Task GeneralizationVictor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach et al.ICLR 2022 · 1,976 citations
  3. Large Language Models Can Be Easily Distracted by Irrelevant ContextFreda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales et al.ICML 2023 · 970 citations
  4. Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis et al.EMNLP 2020 · 142 citations
  5. Generate rather than Retrieve: Large Language Models are Strong Context GeneratorsWenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu et al.ICLR 2023 · 86 citations

Related papers

  1. Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question AnsweringZhanghao Hu, Hanqi Yan, Qinglin Zhu, Zhenyi Shen et al.ACL 2025
  2. OAD-Promoter: Enhancing Zero-Shot VQA Using Large Language Models with Object Attribute DescriptionQuanxing Xu, Ling Zhou, Feifei Zhang, Rubing Huang et al.AAAI 2026
  3. Distilling Knowledge from Reader to Retriever for Question AnsweringGautier Izacard, Edouard GraveICLR 2021 · 317 citations
  4. Improving Passage Retrieval with Zero-Shot Question GenerationDevendra Singh Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan et al.EMNLP 2022 · 69 citations
  5. Retrieval-Augmented Retrieval: Large Language Models are Strong Zero-Shot RetrieverTao Shen, Guodong Long, Xiubo Geng, Chongyang Tao et al.ACL Findings 2024 · 29 citations
Dusk over the sea between two cliffs drawn in fine vertical lines