More Capable, Less Cooperative? When LLMs Fail at Zero-Cost Collaboration
Advait Yadav, Sidney Black, Oliver Sourbut
Abstract
Large language model (LLM) agents increasingly coordinate in multi-agent systems, yet we lack an understanding of where and why cooperation fails. Many real-world coordination problems are not social dilemmas: helping others—sharing documentation, unblocking a teammate—costs the helper almost nothing while producing substantial collective benefit. Whether LLM agents cooperate in this regime, where helping is free and they are explicitly instructed to do so, remains unknown. We build a turn-based multi-agent environment that strips away all strategic complexity, making cooperation costless and trivially optimal. Across eight widely used LLMs, capability does not predict cooperation: OpenAI o3 reaches only 17% of optimal collective performance while the weaker o3-mini reaches 50%, despite identical instructions to maximize group revenue. Using a causal decomposition that automates one side of agent communication, we separate cooperation failures from competence failures, and find that several capable models actively withhold information despite gaining nothing from withholding. Targeted interventions address each mode: explicit protocols roughly double the performance of competence-limited models, while small sharing incentives unlock cooperation-limited ones. Our results suggest that scaling intelligence alone will not solve coordination in multi-agent systems, and will require deliberate cooperative design, even when helping costs nothing.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 9e72ca1f-2a48-42fd-a24e-8f36c74ab398Builds on5
- Generative Agents: Interactive Simulacra of Human BehaviorJoon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris et al.UIST 2023 · 1,882 citations
- Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM AgentsGiorgio Piatti, Zhijing Jin, Max Kleiman-Weiner, Bernhard Schölkopf et al.NeurIPS 2024 · 151 citations
- Learning Efficient Multi-agent Communication: An Information Bottleneck ApproachRundong Wang, Xu He, Runsheng Yu, Wei Qiu et al.ICML 2020 · 133 citations
- Observation Interference in Partially Observable Assistance GamesScott Emmons, Caspar Oesterheld, Vincent Conitzer, Stuart RussellICML 2025
- Cheap Talk Discovery and Utilization in Multi-Agent Reinforcement LearningYat Long Lo, Christian Schröder de Witt, Samuel Sokota, Jakob Nicolaus Foerster et al.ICLR 2023
Related papers
- Spontaneous Giving and Calculated Greed in Language ModelsYuxuan Li, Hirokazu ShiradoEMNLP 2025 · 1 citation
- Hidden in Plain Sight: Reasoning in Underspecified and Misspecified Scenarios for Multimodal LLMsQianqi Yan, Hongquan Li, Shan Jiang, Yang Zhao et al.EMNLP 2025
- Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMsYuxuan Li, Aoi Naito, Hirokazu ShiradoICML 2026 · 9 citations
- SILO-BENCH: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM SystemsYuzhe Zhang, Feiran Liu, Yi Shan, Xinyi Huang et al.ACL 2026 · 5 citations
- Multi-Agent Teams Hold Experts BackAneesh Pappu, Batu El, Hancheng Cao, Carmelo di Nolfo et al.ICML 2026 · 11 citations
