EMNLP Findings2024
MalAlgoQA: Pedagogical Evaluation of Counterfactual Reasoning in Large Language Models and Implications for AI in Education
Shashank Sonkar, Naiming Liu, Myco Le, Richard G. Baraniuk
Abstract
This paper introduces MalAlgoQA, a novel dataset designed to evaluate the counterfactual reasoning capabilities of Large Language Models (LLMs) through a pedagogical approach. The dataset comprises mathematics and reading comprehension questions, each accompanied by four answer choices and their corresponding rationales. At the heart of MalAlgoQA are "malgorithms" -rationales behind incorrect answer choices that represent flawed yet logically coherent reasoning paths. These malgorithms serve as counterfactual scenarios, allowing us to assess an LLM's ability to identify and analyze flawed reasoning patterns. We propose the Malgorithm Identification task, where LLMs are assessed based on their ability to identify corresponding malgorithm given an incorrect answer choice. To evaluate the model performance, we introduce two metrics: Algorithm Identification Accuracy (AIA) for correct answer rationale identification, and Malgorithm Identification Accuracy (MIA) for incorrect answer rationale identification. Our experiments reveal that state-of-the-art LLMs exhibit significant performance drops in MIA compared to AIA, highlighting the challenges in counterfactual reasoning. Surprisingly, we find that the chain-of-thought prompting technique not only fails to consistently enhance MIA but can sometimes lead to underperformance compared to simple prompting. These findings have important implications for developing LLMs with improved counterfactual reasoning, particularly relevant for AI-powered tutoring systems, where identifying and addressing student misconceptions is essential. MalAlgoQA dataset is available here.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Explaining NLP Models via Minimal Contrastive Editing (MiCE) cites Learning The Difference That Makes A Difference With Counterfactually-Augmented Data
- Pedagogical Alignment of Large Language Models cites CLASS: A Design Framework for Building Intelligent Tutoring Systems Based on Learning Science principles
Cited by top-tier papers1
Builds on10
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- Learning The Difference That Makes A Difference With Counterfactually-Augmented DataDivyansh Kaushik, Eduard H. Hovy, Zachary Chase LiptonICLR 2020 · 625 citations
- Explaining NLP Models via Minimal Contrastive Editing (MiCE)Alexis Ross, Ana Marasovic, Matthew E. PetersACL Findings 2021 · 86 citations
- CLASS: A Design Framework for Building Intelligent Tutoring Systems Based on Learning Science principlesShashank Sonkar, Naiming Liu, Debshila Basu Mallick, Richard G. BaraniukEMNLP Findings 2023 · 35 citations
- Pedagogical Alignment of Large Language ModelsShashank Sonkar, Kangqi Ni, Sapana Chaudhary, Richard G. BaraniukEMNLP Findings 2024 · 31 citations
Related papers
- What If the TV was off? Examining Counterfactual Reasoning Abilities of Multi-modal Language ModelsLetian Zhang, Xiaotong Zhai, Zhongkai Zhao, Yongshuo Zong et al.CVPR 2024 · 9 citations
- IfQA: A Dataset for Open-domain Question Answering under Counterfactual PresuppositionsWenhao Yu, Meng Jiang, Peter Clark, Ashish SabharwalEMNLP 2023 · 6 citations
- CounterBench: Evaluating and Improving Counterfactual Reasoning in Large Language ModelsYuefei Chen, Vivek K. Singh, Jing Ma, Ruixiang TangAAAI 2026 · 1 citation
- No Need for Explanations: LLMs can implicitly learn from mistakes in-contextLisa Alazraki, Maximilian Mozes, Jon Ander Campos, Yi Chern Tan et al.EMNLP 2025
- Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-ThoughtAbulhair Saparov, He HeICLR 2023 · 38 citations
