Lune

EMNLP Findings2024

Designing Logic Pattern Templates for Counter-Argument Logical Structure Analysis

Shoichi Naito, Wenzhi Wang, Paul Reisert, Naoya Inoue, Camélia Guerraoui, Kenshi Yamaguchi, Jungmin Choi, Irfan Robbani, Surawat Pothong, Kentaro Inui

Abstract

Counterarguments (CAs) are a valuable tool in education for enhancing critical thinking skills. Despite their effectiveness, the logical attack structure of counterarguments in relation to their corresponding opponent argument remains unexplored due to its complexity. Towards tackling this challenge, in this work, we introduce Counter-Argument Logical Structure Analysis (CALSA), a new task. We first define 10 new CA logic patterns, each comprised of a unique template and slots. We then conduct an annotation study on top of 778 CAs using our patterns to create a new dataset. Our dataset achieves high annotator agreement (Krippendorff α=0.50) and high coverage (86.5%). We perform preliminary experiments employing recent large language models to assess the feasibility of automating CA logical structure analysis. Our findings highlight the task's inherent complexity within a straightforward framework, indicating exciting opportunities for further exploration. We publicly release our dataset and model scripts at https://github.com/cl-tohoku/CALSA.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Identification of Multiple Logical Interpretations in Counter-Arguments cites Architectural Sweet Spots for Modeling Human Label Variation by the Example of Argument Quality: It's Best to Relate Perspectives!
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Identification of Multiple Logical Interpretations in Counter-ArgumentsWenzhi Wang, Paul Reisert, Shoichi Naito, Naoya Inoue et al.EMNLP 2025

Builds on3

  1. APE: Argument Pair Extraction from Peer Review and Rebuttal via Multi-task LearningLiying Cheng, Lidong Bing, Qian Yu, Wei Lu et al.EMNLP 2020 · 56 citations
  2. Out of the Echo Chamber: Detecting Countering Debate SpeechesMatan Orbach, Yonatan Bilu, Assaf Toledo, Dan Lahav et al.ACL 2020 · 1 citation
  3. Architectural Sweet Spots for Modeling Human Label Variation by the Example of Argument Quality: It's Best to Relate Perspectives!Philipp Heinisch, Matthias Orlikowski, Julia Romberg, Philipp CimianoEMNLP 2023 · 1 citation

Related papers

  1. Flee the Flaw: Annotating the Underlying Logic of Fallacious Arguments Through Templates and Slot-fillingIrfan Robbani, Paul Reisert, Surawat Pothong, Naoya Inoue et al.EMNLP 2024
  2. Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language ModelsBlanca Calvo Figueras, Rodrigo AgerriEMNLP Findings 2025
  3. MalAlgoQA: Pedagogical Evaluation of Counterfactual Reasoning in Large Language Models and Implications for AI in EducationShashank Sonkar, Naiming Liu, Myco Le, Richard G. BaraniukEMNLP Findings 2024 · 6 citations
  4. BriefMe: A Legal NLP Benchmark for Assisting with Legal BriefsJesse Woo, Fateme Hashemi Chaleshtori, Ana Marasovic, Kenneth MarinoACL Findings 2025 · 5 citations
  5. CLOMO: Counterfactual Logical Modification with Large Language ModelsYinya Huang, Ruixin Hong, Hongming Zhang, Wei Shao et al.ACL 2024 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines