ACL Findings2026
Identifying Collective Intelligence Factor in LLM Agent Groups for Generalizable Multi-Agent System Design
Zhilun Zhou, Zihan Liu, Jiahe Liu, Yihan Wang, Qingyu Shao, Fengli Xu, Depeng Jin, Yong Li
Abstract
Large language model (LLM)-based multiagent systems (MASs) have shown impressive performance in solving a wide range of complex problems. However, previous studies mainly focus on designing customized MAS for specific tasks, while a critical research problem remains unclear: Do LLM agent groups exhibit a form of "general intelligence" that reflects their general ability across various tasks? Researchers have found a Collective Intelligence (CI) factor in human groups that captures their general capability. Inspired by this, in this study, we aim to investigate whether an analogous CI factor also exists in LLM agent groups, which is crucial for building generalizable MAS. Motivated by human cognitive psychology experiments, we construct 108 LLM agent groups with diverse group sizes, LLM compositions, and communication topologies. We systematically evaluate these groups across a wide range of tasks and analyze their performances. Our results demonstrate that an Artificial Collective Intelligence (ACI) factor can be extracted from LLM agent groups to predict the generalization performance on new tasks. Inspired by this, we train a model to predict the ACI based on the features of MAS, and show that it can be used as a plug-in to enhance the generalization ability of MAS optimization methods.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies cites Improving Factuality and Reasoning in Language Models through Multiagent Debate
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors cites CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors cites Improving Factuality and Reasoning in Language Models through Multiagent Debate
- Multi-LLM Debate: Framework, Principals, and Interventions cites Improving Factuality and Reasoning in Language Models through Multiagent Debate
Builds on12
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model SocietyGuohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin et al.NeurIPS 2023 · 1,975 citations
- Improving Factuality and Reasoning in Language Models through Multiagent DebateYilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum et al.ICML 2024 · 1,562 citations
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent BehaviorsWeize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang et al.ICLR 2024 · 594 citations
- Multi-LLM Debate: Framework, Principals, and InterventionsAndrew Estornell, Yang LiuNeurIPS 2024 · 131 citations
- Multi-Agent Design: Optimizing Agents with Better Prompts and TopologiesHan Zhou, Xingchen Wan, Ruoxi Sun, Hamid Palangi et al.ICLR 2026 · 127 citations
Related papers
- OMAC: A Holistic Optimization Framework for LLM-Based Multi-Agent CollaborationShijun Li, Hilaf Hasson, Joydeep GhoshICML 2026
- ResMAS: Resilience Optimization in LLM-based Multi-agent SystemsZhilun Zhou, Zihan Liu, Jiahe Liu, Qingyu Shao et al.AAAI 2026 · 2 citations
- Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative AgentsHaochen Sun, Shuwen Zhang, Lujie Niu, Lei Ren et al.EMNLP 2025
- Measuring and Mitigating Rapport Bias of Large Language Models under Multi-Agent Social InteractionsMaojia Song, Pala Tej Deep, Ruiwen Zhou, Weisheng Jin et al.ICLR 2026
- G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural NetworksGuibin Zhang, Yanwei Yue, Xiangguo Sun, Guancheng Wan et al.ICML 2025
