EMNLP Findings2021
Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data
Dian Yu, Kai Sun, Dong Yu, Claire Cardie
Abstract
Despite considerable progress, most machine reading comprehension (MRC) tasks still lack sufficient training data to fully exploit powerful deep neural network models with millions of parameters, and it is laborious, expensive, and time-consuming to create largescale, high-quality MRC data through crowdsourcing. This paper focuses on generating more training data for MRC tasks by leveraging existing question-answering (QA) data. We first collect a large-scale multi-subject multiple-choice QA dataset for Chinese, Ex-amQA. We next use incomplete, yet relevant snippets returned by a web search engine as the context for each QA instance to convert it into a weakly-labeled MRC instance. To better use the weakly-labeled data to improve a target MRC task, we evaluate and compare several methods and further propose a self-teaching paradigm. Experimental results show that, upon state-of-the-art MRC baselines, we can obtain +5.1% in accuracy on a multiple-choice Chinese MRC dataset, C 3 , and +3.8% in exact match on an extractive Chinese MRC dataset, CMRC 2018, demonstrating the usefulness of the generated QAbased weakly-labeled data for different types of MRC tasks as well as the effectiveness of self-teaching. ExamQA will be available at https://dataset.org/examqa/.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Revisiting Pre-Trained Models for Chinese Natural Language Processing cites ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding
- AMBERT: A Pre-trained Language Model with Multi-Grained Tokenization cites ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding
Cited by top-tier papers1
Builds on12
- ERNIE 2.0: A Continual Pre-Training Framework for Language UnderstandingYu Sun, Shuohuan Wang, Yu-Kun Li, Shikun Feng et al.AAAI 2020 · 885 citations
- Revisiting Pre-Trained Models for Chinese Natural Language ProcessingYiming Cui, Wanxiang Che, Ting Liu, Bing Qin et al.EMNLP Findings 2020 · 866 citations
- Revisiting Self-Training for Neural Sequence GenerationJunxian He, Jiatao Gu, Jiajun Shen, Marc'Aurelio RanzatoICLR 2020 · 294 citations
- JEC-QA: A Legal-Domain Question Answering DatasetHaoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang et al.AAAI 2020 · 212 citations
- AMBERT: A Pre-trained Language Model with Multi-Grained TokenizationXinsong Zhang, Pengshuai Li, Hang LiACL Findings 2021 · 54 citations
Related papers
- Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain TextDongfang Li, Baotian Hu, Qingcai Chen, Weihua Peng et al.EMNLP 2020 · 39 citations
- A Self-Training Method for Machine Reading Comprehension with Soft Evidence ExtractionYilin Niu, Fangkai Jiao, Mantong Zhou, Ting Yao et al.ACL 2020 · 33 citations
- MMM: Multi-Stage Multi-Task Learning for Multi-Choice Reading ComprehensionDi Jin, Shuyang Gao, Jiun-Yu Kao, Tagyoung Chung et al.AAAI 2020 · 72 citations
- Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and ResourcesTaolin Zhang, Chengyu Wang, Minghui Qiu, Bite Yang et al.ACL Findings 2021 · 2 citations
- Natural Response Generation for Chinese Reading ComprehensionNuo Chen, Hongguang Li, Yinan Bao, Baoyuan Wang et al.EMNLP Findings 2023 · 2 citations
