ACL Findings2025
Can Large Language Models Understand Argument Schemes?
Elfia Bezou-Vrakatseli, Oana Cocarascu, Sanjay Modgil
Abstract
Argument schemes represent stereotypical patterns of reasoning that occur in everyday arguments.However, despite their usefulness, argument scheme classification, that is classifying natural language arguments according to the schemes they are instances of, is an underexplored task in NLP.In this paper we present a systematic evaluation of large language models (LLMs) for classifying argument schemes based on Walton's taxonomy.We experiment with seven LLMs in zero-shot, few-shot, and chain-of-thought prompting, and explore two strategies to enhance task instructions: employing formal definitions and LLM-generated descriptions.Our analysis on both manually annotated and automatically generated arguments, including enthymemes, indicates that while larger models exhibit satisfactory performance in identifying argument schemes, challenges remain for smaller models.Our work offers the first comprehensive assessment of LLMs in identifying argument schemes, and provides insights for advancing reasoning capabilities in computational argumentation.Argument from Consequences P: If A is brought about, good/bad consequences occur.C: Therefore, A should (not) be brought about.Argument from Example P: In this particular case, a has properties F and G. C: So, generally, if x has property F , then it also has G. Argument from Values P1: Value V is positive/negative as judged by agent a. P2: If V is positive it is a reason to commit to goal G. P2': If V is negative it is a reason to not commit to goal G. C: V is a reason for agent a to (not) commit to goal G. Argument from Cause to Effect P1: Generally, if A occurs, then B will (might) occur.P2: In this case, A occurs (might occur).C: Therefore, in this case, B will (might) occur.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on10
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- A Survey on In-context LearningQingxiu Dong, Lei Li, Damai Dai, Ce Zheng et al.EMNLP 2024 · 479 citations
- MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical NotesAsma Ben Abacha, Wen-Wai Yim, Yujuan Fu, Zhaoyi Sun et al.ACL Findings 2025 · 29 citations
- Exploring the Potential of Large Language Models in Computational ArgumentationGuizhen Chen, Liying Cheng, Anh Tuan Luu, Lidong BingACL 2024 · 8 citations
Related papers
- Are LLMs Good Zero-Shot Fallacy Classifiers?Fengjun Pan, Xiaobao Wu, Zongrui Li, Anh Tuan LuuEMNLP 2024 · 7 citations
- Beyond Recognising Entailment: Formalising Natural Language Inference from an Argumentative PerspectiveAmeer Saadat-Yazdi, Nadin KökciyanACL 2024
- Mining Complex Patterns of Argumentative Reasoning in Natural Language DialogueRamon Ruiz-Dolz, Zlata Kikteva, John LawrenceACL 2025 · 2 citations
- Can LLMs Judge Debates? Evaluating Non-Linear Reasoning via Argumentation Theory SemanticsReza Sanayei, Srdjan Vesic, Eduardo Blanco, Mihai SurdeanuEMNLP Findings 2025 · 2 citations
- ArgU: A Controllable Factual Argument GeneratorSougata Saha, Rohini K. SrihariACL 2023 · 4 citations
