Lune

EMNLP Findings2024

MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate

Alfonso Amayuelas, Xianjun Yang, Antonis Antoniades, Wenyue Hua, Liangming Pan, William Yang Wang

Abstract

Large Language Models (LLMs) have shown exceptional results on current benchmarks when working individually. The advancement in their capabilities, along with a reduction in parameter size and inference times, has facilitated the use of these models as agents, enabling interactions among multiple models to execute complex tasks. Such collaborations offer several advantages, including the use of specialized models (e.g. coding), improved confidence through multiple computations, and enhanced divergent thinking, leading to more diverse outputs. Thus, the collaborative use of language models is expected to grow significantly in the coming years. In this work, we evaluate the behavior of a network of models collaborating through debate under the influence of an adversary. We introduce pertinent metrics to assess the adversary's effectiveness, focusing on system accuracy and model agreement. Our findings highlight the importance of a model's persuasive ability in influencing others. Additionally, we explore inference-time methods to generate more compelling arguments and evaluate the potential of promptbased mitigation as a defensive strategy.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

16 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Models cites Improving Factuality and Reasoning in Language Models through Multiagent Debate
  • Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Models cites ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
  • TruthfulQA: Measuring How Models Mimic Human Falsehoods cites Measuring Massive Multitask Language Understanding
  • GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modeling cites Measuring Massive Multitask Language Understanding
  • GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modeling cites Improving Factuality and Reasoning in Language Models through Multiagent Debate
  • GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modeling cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Agents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks cites Improving Factuality and Reasoning in Language Models through Multiagent Debate
  • Agents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks cites ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
  • Improving Factuality and Reasoning in Language Models through Multiagent Debate cites Measuring Massive Multitask Language Understanding
  • Improving Factuality and Reasoning in Language Models through Multiagent Debate cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat Detection cites Measuring Massive Multitask Language Understanding
  • SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat Detection cites TruthfulQA: Measuring How Models Mimic Human Falsehoods
  • SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat Detection cites Improving Factuality and Reasoning in Language Models through Multiagent Debate
  • SAGA: A Security Architecture for Governing AI Agentic Systems cites Agents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks
  • ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate cites Improving Factuality and Reasoning in Language Models through Multiagent Debate
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers16

  1. SAGA: A Security Architecture for Governing AI Agentic SystemsGeorgios Syros, Anshuman Suri, Jacob Ginesin, Cristina Nita-Rotaru et al.NDSS 2026 · 63 citations
  2. GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph ModelingJialong Zhou, Lichao Wang, Xiao YangNeurIPS 2025 · 40 citations
  3. Agents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt AttacksRana Muhammad Shahroz, Zhen Tan, Sukwon Yun, Charles Fleming et al.ACL 2025 · 18 citations
  4. SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat DetectionYang Feng, Xudong PanWWW 2026 · 3 citations
  5. Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language ModelsChen Han, Wenzhen Zheng, Xijin TangEMNLP 2025 · 2 citations

Builds on11

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
  3. TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
  4. Improving Factuality and Reasoning in Language Models through Multiagent DebateYilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum et al.ICML 2024 · 1,562 citations
  5. ChatEval: Towards Better LLM-based Evaluators through Multi-Agent DebateChi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu et al.ICLR 2024 · 871 citations

Related papers

  1. Collaborative Reasoner: Self-Improving Social Agents with Synthetic ConversationsAnsong Ni, Ruta Desai, Yang Li, Xinjie Lei et al.NeurIPS 2025 · 7 citations
  2. Social Dynamics as Critical Vulnerabilities that Undermine Objective Decision-Making in LLM CollectivesChanggeon Ko, Jisu Shin, Hoyun Song, Huije Lee et al.ACL 2026 · 1 citation
  3. Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology ViewJintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu et al.ACL 2024
  4. Multi-Agent Debate for LLM Judges with Adaptive Stability DetectionTianyu Hu, Zhen Tan, Song Wang, Huaizhi Qu et al.NeurIPS 2025 · 25 citations
  5. ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM CollaborationAndrew Estornell, Jean-Francois Ton, Yuanshun Yao, Yang LiuICLR 2025 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines