Lune

ACL Findings2024

Unsupervised Distractor Generation via Large Language Model Distilling and Counterfactual Contrastive Decoding

Fanyi Qu, Hao Sun, Yunfang Wu

Abstract

Within the context of reading comprehension, the task of Distractor Generation (DG) aims to generate several incorrect options to confuse readers. Traditional supervised methods for DG rely heavily on expensive human-annotated distractor labels. In this paper, we propose an unsupervised DG framework, leveraging Large Language Models (LLMs) as cost-effective annotators to enhance the DG capability of smaller student models. Specially, to perform knowledge distilling, we propose a dual task training strategy that integrates pseudo distractors from LLMs and the original answer information as the objective targets with a twostage training process. Moreover, we devise a counterfactual contrastive decoding mechanism for increasing the distracting capability of the DG model. Experiments show that our unsupervised generation method with Bart-base greatly surpasses GPT-3.5-turbo performance with only 200× fewer model parameters. Our proposed unsupervised DG method offers a cost-effective framework for practical reading comprehension applications, without the need of laborious distractor annotation and costly large-size models.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Generating Plausible Distractors for Multiple-Choice Questions via Student Choice Prediction cites DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions
  • Generating Plausible Distractors for Multiple-Choice Questions via Student Choice Prediction cites Knowledge-Driven Distractor Generation for Cloze-Style Multiple Choice Questions
  • Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation cites Knowledge-Driven Distractor Generation for Cloze-Style Multiple Choice Questions
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and EvaluationElaf Alhazmi, Quan Sheng, Wei Emma Zhang, Munazza Zaib et al.EMNLP 2024 · 15 citations
  2. DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice QuestionsNigel Fernandez, Alexander Scarlatos, Wanyong Feng, Simon Woodhead et al.EMNLP 2024 · 10 citations
  3. Generating Plausible Distractors for Multiple-Choice Questions via Student Choice PredictionYooseop Lee, Suin Kim, Yohan JoACL 2025
  4. HumanVBench: Probing Human-Centric Video Understanding in MLLMs with Automatically Synthesized BenchmarksTing Zhou, Daoyuan Chen, Qirui Jiao, Bolin Ding et al.CVPR 2026

Builds on9

  1. Want To Reduce Labeling Cost? GPT-3 Can HelpShuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu et al.EMNLP Findings 2021 · 177 citations
  2. Contrastive Decoding: Open-ended Text Generation as OptimizationXiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang et al.ACL 2023 · 78 citations
  3. Ask Me Anything: A simple strategy for prompting language modelsSimran Arora, Avanika Narayan, Mayee F. Chen, Laurel J. Orr et al.ICLR 2023 · 74 citations
  4. Knowledge-Driven Distractor Generation for Cloze-Style Multiple Choice QuestionsSiyu Ren, Kenny Q. ZhuAAAI 2021 · 62 citations
  5. LLM-powered Data Augmentation for Enhanced Cross-lingual PerformanceChenxi Whitehouse, Monojit Choudhury, Alham Fikri AjiEMNLP 2023 · 53 citations

Related papers

  1. DisGeM: Distractor Generation for Multiple Choice Questions with Span MaskingDevrim Çavusoglu, Seçil Sen, Ulas SertEMNLP Findings 2024 · 6 citations
  2. Fine-Tuning Encoder-Decoder Models with Contrastive Learning for In-Context Distractor GenerationElaf Alhazmi, Quan Z. Sheng, Wei Emma Zhang, Mohammed I. Thanoon et al.EMNLP Findings 2025
  3. Knowledge Infused DecodingRuibo Liu, Guoqing Zheng, Shashank Gupta, Radhika Gaonkar et al.ICLR 2022 · 18 citations
  4. Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question AnsweringSukmin Cho, Jeongyeon Seo, Soyeong Jeong, Jong C. ParkEMNLP Findings 2023 · 3 citations
  5. Enhancing Distractor Generation for Multiple-Choice Questions with Retrieval Augmented Pretraining and Knowledge Graph IntegrationHan-Cheng Yu, Yu-An Shih, Kin-Man Law, Kai-Yu Hsieh et al.ACL Findings 2024 · 18 citations
Dusk over the sea between two cliffs drawn in fine vertical lines