Lune

EMNLP Findings2020

Regularization of Distinct Strategies for Unsupervised Question Generation

Junmo Kang, Giwon Hong, Haritz Puerto San Roman, Sung-Hyon Myaeng

Abstract

Unsupervised question answering (UQA) has been proposed to avoid the high cost of creating high-quality datasets for QA. One approach to UQA is to train a QA model with questions generated automatically. However, the generated questions are either too similar to a word sequence in the context or too drifted from the semantics of the context, thereby making it difficult to train a robust QA model. We propose a novel regularization method based on teacher-student architecture to avoid bias toward a particular question generation strategy and modulate the process of generating individual words when a question is generated. Our experiments demonstrate that we have achieved the goal of generating higher-quality questions for UQA across diverse QA datasets and tasks. We also show that this method can be useful for creating a QA model with few-shot learning.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Distill or Annotate? Cost-Efficient Fine-Tuning of Compact ModelsJunmo Kang, Wei Xu, Alan RitterACL 2023 · 5 citations
  2. Elaboration-Generating Commonsense Question Answering at ScaleWenya Wang, Vivek Srikumar, Hannaneh Hajishirzi, Noah A. SmithACL 2023 · 5 citations

Builds on1

  1. Training Question Answering Models From Synthetic DataRaul Puri, Ryan Spring, Mohammad Shoeybi, Mostofa Patwary et al.EMNLP 2020 · 15 citations

Related papers

  1. Harvesting and Refining Question-Answer Pairs for Unsupervised QAZhongli Li, Wenhui Wang, Li Dong, Furu Wei et al.ACL 2020 · 29 citations
  2. Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word GenerationQifan Wang, Li Yang, Xiaojun Quan, Fuli Feng et al.EMNLP 2022 · 5 citations
  3. AttenWalker: Unsupervised Long-Document Question Answering via Attention-based Graph WalkingYuxiang Nie, Heyan Huang, Wei Wei, Xian-Ling MaoACL Findings 2023 · 2 citations
  4. AutoEQA: Auto-Encoding Questions for Extractive Question AnsweringStalin Varanasi, Saadullah Amin, Guenter NeumannEMNLP Findings 2021 · 2 citations
  5. An Empirical Comparison of LM-based Question and Answer Generation MethodsAsahi Ushio, Fernando Alva-Manchego, José Camacho-ColladosACL Findings 2023 · 25 citations
Dusk over the sea between two cliffs drawn in fine vertical lines