Lune

ACL Findings2023

SkillQG: Learning to Generate Question for Reading Comprehension Assessment

Xiaoqiang Wang, Bang Liu, Siliang Tang, Lingfei Wu

Abstract

We present SkillQG: a question generation framework with controllable comprehension types for assessing and improving machine reading comprehension models.Existing question generation systems widely differentiate questions by literal information such as question words and answer types to generate semantically relevant questions for a given context.However, they rarely consider the comprehension nature of questions, i.e. the different comprehension capabilities embodied by different questions.In comparison, our SkillQG is able to tailor a fine-grained assessment and improvement to the capabilities of question answering models built on it.Specifically, we first frame the comprehension type of questions based on a hierarchical skill-based schema, then formulate SkillQG as a skill-conditioned question generator.Furthermore, to improve the controllability of generation, we augment the input text with question focus and skill-specific knowledge, which are constructed by iteratively prompting the pre-trained language models.Empirical results demonstrate that SkillQG outperforms baselines in terms of quality, relevance, and skill-controllability while showing a promising performance boost in downstream question answering task.† Canada CIFAR AI Chair.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • BARTScore: Evaluating Generated Text as Text Generation cites BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
  • BARTScore: Evaluating Generated Text as Text Generation cites BERTScore: Evaluating Text Generation with BERT
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on24

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  3. The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations
  4. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
  5. BARTScore: Evaluating Generated Text as Text GenerationWeizhe Yuan, Graham Neubig, Pengfei LiuNeurIPS 2021 · 1,143 citations

Related papers

  1. Question Generation for Reading Comprehension Assessment by Modeling How and What to AskBilal Ghanem, Lauren Lutz Coleman, Julia Rivard Dexter, Spencer McIntosh von der Ohe et al.ACL Findings 2022 · 25 citations
  2. Learning to Generate Questions by Learning to Recover Answer-containing SentencesSeohyun Back, Akhil Kedia, Sai Chetan Chinthakindi, Haejun Lee et al.ACL Findings 2021 · 10 citations
  3. Asking Questions the Human Way: Scalable Question-Answer Generation from Text CorpusBang Liu, Haojie Wei, Di Niu, Haolan Chen et al.WWW 2020 · 100 citations
  4. Fantastic Questions and Where to Find Them: FairytaleQA - An Authentic Dataset for Narrative ComprehensionYing Xu, Dakuo Wang, Mo Yu, Daniel Ritchie et al.ACL 2022 · 131 citations
  5. Generating Well-Formed Answers by Machine Reading with Stochastic Selector NetworksBin Bi, Chen Wu, Ming Yan, Wei Wang et al.AAAI 2020 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines