Lune

ACL Findings2023

When to Read Documents or QA History: On Unified and Selective Open-domain QA

Kyungjae Lee, Sang-eun Han, Seung-won Hwang, Moontae Lee

Abstract

This paper studies the problem of open-domain question answering, with the aim of answering a diverse range of questions leveraging knowledge resources. Two types of sources, QApair and document corpora, have been actively leveraged with the following complementary strength. The former is highly precise when the paraphrase of given question q was seen and answered during training, often posed as a retrieval problem, while the latter generalizes better for unseen questions. A natural follow-up is thus leveraging both models, while a naive pipelining or integration approaches have failed to bring additional gains over either model alone. Our distinction is interpreting the problem as calibration, which estimates the confidence of predicted answers as an indicator to decide when to use a document or QA-pair corpus. The effectiveness of our method was validated on widely adopted benchmarks such as Natural Questions and TriviaQA.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Distilling Knowledge from Reader to Retriever for Question Answering cites Dense Passage Retrieval for Open-Domain Question Answering
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on12

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Retrieval Augmented Language Model Pre-TrainingKelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat et al.ICML 2020 · 2,937 citations
  3. Revisiting the Calibration of Modern Neural NetworksMatthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis et al.NeurIPS 2021 · 633 citations
  4. Distilling Knowledge from Reader to Retriever for Question AnsweringGautier Izacard, Edouard GraveICLR 2021 · 317 citations
  5. Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis et al.EMNLP 2020 · 142 citations

Related papers

  1. You Only Need One Model for Open-domain Question AnsweringHaejun Lee, Akhil Kedia, Jongwon Lee, Ashwin Paranjape et al.EMNLP 2022
  2. Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting EvidenceHung-Ting Chen, Michael J. Q. Zhang, Eunsol ChoiEMNLP 2022 · 27 citations
  3. Knowing More About Questions Can Help: Improving Calibration in Question AnsweringShujian Zhang, Chengyue Gong, Eunsol ChoiACL Findings 2021 · 25 citations
  4. Open-domain Question Answering via Chain of Reasoning over Heterogeneous KnowledgeKaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg et al.EMNLP Findings 2022 · 16 citations
  5. Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence AnnotationChen Zhao, Chenyan Xiong, Jordan L. Boyd-Graber, Hal Daumé IIIEMNLP 2021 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines