ACL Findings2025
Do Emotions Really Affect Argument Convincingness? A Dynamic Approach with LLM-based Manipulation Checks
Abstract
Emotions have been shown to play a role in argument convincingness, yet this aspect is underexplored in the natural language processing (NLP) community. Unlike prior studies that use static analyses, focus on a single text domain or language, or treat emotion as just one of many factors, we introduce a dynamic framework inspired by manipulation checks commonly used in psychology and social science; leveraging LLM-based manipulation checks, this framework examines the extent to which perceived emotional intensity influences perceived convincingness. Through human evaluation of arguments across different languages, text domains, and topics, we find that in over half of cases, human judgments of convincingness remain unchanged despite variations in perceived emotional intensity; when emotions do have an impact, they more often enhance rather than weaken convincingness. We further analyze whether 11 LLMs behave like humans in the same scenario, finding that while LLMs generally mirror human patterns, they struggle to capture nuanced emotional effects in individual judgments.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Cognitive Bias in Decision-Making with LLMs cites Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
- Cognitive Bias in Decision-Making with LLMs cites Towards Understanding and Mitigating Social Biases in Language Models
- Towards Understanding and Mitigating Social Biases in Language Models cites Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Builds on9
- Towards Understanding and Mitigating Social Biases in Language ModelsPaul Pu Liang, Chiyu Wu, Louis-Philippe Morency, Ruslan SalakhutdinovICML 2021 · 495 citations
- Reducing Sentiment Bias in Language Models via Counterfactual EvaluationPo-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth et al.EMNLP Findings 2020 · 124 citations
- Cognitive Bias in Decision-Making with LLMsJessica Maria Echterhoff, Yao Liu, Abeer Alessa, Julian J. McAuley et al.EMNLP Findings 2024 · 62 citations
- Logical Fallacy DetectionZhijing Jin, Abhinav Lalwani, Tejas Vaidhya, Xiaoyu Shen et al.EMNLP Findings 2022 · 52 citations
- Can Language Models Recognize Convincing Arguments?Paula Rescala, Manoel Horta Ribeiro, Tiancheng Hu, Robert WestEMNLP Findings 2024 · 12 citations
Related papers
- "I understand your perspective": LLM Persuasion through the Lens of Communicative Action TheoryEsra Dönmez, Agnieszka FalenskaACL Findings 2025 · 1 citation
- EAI: Emotional Decision-Making of LLMs in Strategic Games and Ethical DilemmasMikhail Mozikov, Nikita Severin, Valeria Bodishtianu, Maria Glushanina et al.NeurIPS 2024 · 21 citations
- Do Emotions Influence Moral Judgment in Large Language Models?Mohammad Saim, Tianyu JiangACL Findings 2026
- AI Argues Differently: Distinct Argumentative and Linguistic Patterns of LLMs in Persuasive ContextsEsra Dönmez, Maximilian Maurer, Gabriella Lapesa, Agnieszka FalenskaEMNLP 2025 · 1 citation
- The Bots of Persuasion: Examining How Conversational Agents' Linguistic Expressions of Personality Affect User Perceptions and DecisionsHüseyin Ugur Genç, Heng Gu, Chadha Degachi, Evangelos Niforatos et al.CHI 2026 · 1 citation
