EMNLP Findings2024
Improving Argument Effectiveness Across Ideologies using Instruction-tuned Large Language Models
Roxanne El Baff, Khalid Al Khatib, Milad Alshomary, Kai Konen, Benno Stein, Henning Wachsmuth
Abstract
Different political ideologies (e.g., liberal and conservative Americans) hold different worldviews, which leads to opposing stances on different issues (e.g., gun control) and, thereby, fostering societal polarization. Arguments are a means of bringing the perspectives of people with different ideologies closer together, depending on how well they reach their audience. In this paper, we study how to computationally turn ineffective arguments into effective arguments for people with certain ideologies by using instruction-tuned large language models (LLMs), looking closely at style features. For development and evaluation, we collect ineffective arguments per ideology from debate.org, and we generate about 30k, which we rewrite using three LLM methods tailored to our task: zero-shot prompting, few-shot prompting, and LLM steering. Our experiments provide evidence that LLMs naturally improve argument effectiveness for liberals. Our LLM-based and human evaluation show a clear preference towards the rewritten arguments. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on11
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Whose Opinions Do Language Models Reflect?Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee et al.ICML 2023 · 764 citations
- M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge DistillationJianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo et al.ACL Findings 2024 · 762 citations
- Prometheus: Inducing Fine-Grained Evaluation Capability in Language ModelsSeungone Kim, Jamin Shin, Yejin Choi, Joel Jang et al.ICLR 2024 · 468 citations
- A Large-Scale Dataset for Argument Quality Ranking: Construction and AnalysisShai Gretz, Roni Friedman, Edo Cohen-Karlik, Assaf Toledo et al.AAAI 2020 · 148 citations
Related papers
- LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine FeedbackTimon Ziegenbein, Gabriella Skitalinskaya, Alireza Bayat Makou, Henning WachsmuthACL 2024
- Decomposing Argumentative Essay Generation via Dialectical Planning of Complex ReasoningYuhang He, Jianzhu Bao, Yang Sun, Bin Liang et al.ACL Findings 2024 · 4 citations
- Persona-Assigned Large Language Models Exhibit Human-Like Motivated ReasoningSaloni Dash, Amélie Reymond, Emma S. Spiro, Aylin CaliskanACL Findings 2026 · 1 citation
- How Susceptible are Large Language Models to Ideological Manipulation?Kai Chen, Zihao He, Jun Yan, Taiwei Shi et al.EMNLP 2024 · 4 citations
- Linear Representations of Political Perspective Emerge in Large Language ModelsJunsol Kim, James Evans, Aaron ScheinICLR 2025
