Lune

ACL Findings2026

Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design

Xu Guo, Qiming Ge, Jian Tong, Kedi Chen, Jin Zhang, Xiaogui Yang, Xuan Gao, Haijun Lv, Zhihui Lu, Yicheng Zou, Qipeng Guo

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) significantly enhances the reasoning capabilities of Large Language Models. When applied to RLVR, Multiple-Choice Questions (MCQs) offer a scalable source of verifiable data but risk inducing reward hacking, where models shortcut reasoning via random guessing or simple elimination. Current approaches often mitigate this by converting MCQs to open-ended formats, thereby discarding the contrastive signal provided by expertdesigned distractors. In this work, we systematically investigate the impact of option design on RLVR. Our analysis highlights two primary insights: (1) Mismatches in option counts between training and testing degrade performance.

(2) Strong distractors effectively mitigate random guessing, enabling effective RLVR training even with 2-way questions. Motivated by these findings, we propose Iterative Distractor Curation (IDC), a framework that actively constructs high-quality distractors to block elimination shortcuts and promote deep reasoning. Experiments on various benchmarks demonstrate that our method effectively enhances distractor quality and yields significant gains in RLVR training compared to the original data.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions cites CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. A Survey of Inductive Reasoning for Large Language ModelsKedi Chen, Dezhao Ruan, Yuhao Dan, Yaoting Wang et al.ACL 2026 · 5 citations

Builds on8

  1. SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
  2. Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base ModelJingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang et al.NeurIPS 2025 · 533 citations
  3. Data Augmentation using LLMs: Data Perspectives, Learning Paradigms and ChallengesBosheng Ding, Chengwei Qin, Ruochen Zhao, Tianze Luo et al.ACL Findings 2024 · 106 citations
  4. CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language ModelShang-Hsuan Chiang, Ssu-Cheng Wang, Yao-Chung FanEMNLP Findings 2022 · 12 citations
  5. DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice QuestionsNigel Fernandez, Alexander Scarlatos, Wanyong Feng, Simon Woodhead et al.EMNLP 2024 · 10 citations

Related papers

  1. RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model ReasoningYukun Chen, Jiaming Li, Longze Chen, Ze Gong et al.ICML 2026 · 5 citations
  2. Diversity-Incentivized Exploration for Versatile ReasoningZican Hu, Shilin Zhang, Yafu Li, Jianhao Yan et al.ICLR 2026 · 32 citations
  3. DARL: Encouraging Diverse Answers for General Reasoning without VerifiersChongxuan Huang, Lei Lin, Xiaodong Shi, Wenping Hu et al.ACL Findings 2026
  4. Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious RewardPeter Chen, Xiaopeng Li, Ziniu Li, Wotao Yin et al.ICLR 2026 · 28 citations
  5. Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang et al.NeurIPS 2025 · 1,109 citations
Dusk over the sea between two cliffs drawn in fine vertical lines