ACL Findings2025
CROSSAGENTIE: Cross-Type and Cross-Task Multi-Agent LLM Collaboration for Zero-Shot Information Extraction
Abstract
Large language models (LLMs) excel in generating unstructured text. However, they struggle with producing structured output while maintaining accuracy in zero-shot information extraction (IE), such as named entity recognition (NER) and relation extraction (RE). To address these challenges, we propose CROSSAGEN-TIE, a multi-agent framework that enhances zero-shot IE through multi-agent LLM collaboration. CROSSAGENTIE refines LLM predictions iteratively through two mechanisms: intra-group cross-type debate, which resolves entity-label conflicts through context-based evidence and confidence aggregation, and intergroup cross-task debate, where NER and RE mutually refine outputs via bidirectional feedback. Furthermore, we introduce template finetuning, distilling high-confidence multi-agent outputs into a single model, significantly reducing inference costs while preserving accuracy. Experiments across five NER and five RE datasets show that CROSSAGENTIE significantly outperforms state-of-the-art zero-shot baselines by a large margin. CROSSAGEN-TIE effectively addresses LLM limitations in structured prediction with an effective and efficient approach for zero-shot information extraction. Our GitHub can be found at https://github.com/Luca/CorssAgentIE. Sentence: "Reagan sounded positive notes reminiscent of earlier speeches throughout his political career _ the pre-eminent position of " We the People "in ....
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Multi-LLM Debate: Framework, Principals, and Interventions cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- GPT-RE: In-context Learning for Relation Extraction using Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- GPT-RE: In-context Learning for Relation Extraction using Large Language Models cites Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Builds on22
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
- Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe et al.EMNLP 2022 · 634 citations
- GoLLIE: Annotation Guidelines improve Zero-Shot Information-ExtractionOscar Sainz, Iker García-Ferrero, Rodrigo Agerri, Oier Lopez de Lacalle et al.ICLR 2024 · 168 citations
- GPT-RE: In-context Learning for Relation Extraction using Large Language ModelsZhen Wan, Fei Cheng, Zhuoyuan Mao, Qianying Liu et al.EMNLP 2023 · 132 citations
- Multi-LLM Debate: Framework, Principals, and InterventionsAndrew Estornell, Yang LiuNeurIPS 2024 · 131 citations
Related papers
- A Cooperative Multi-Agent Framework for Zero-Shot Named Entity RecognitionZihan Wang, Ziqi Zhao, Yougang Lyu, Zhumin Chen et al.WWW 2025 · 16 citations
- Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument ExtractionKuan-Hao Huang, I-Hung Hsu, Prem Natarajan, Kai-Wei Chang et al.ACL 2022
- Extracting Events Like Code: A Multi-Agent Programming Framework for Zero-Shot Event ExtractionQuanjiang Guo, Sijie Wang, Jinchuan Zhang, Ben Zhang et al.AAAI 2026
- A Multi-Agent LLM Framework for Multi-Domain Low-Resource In-Context NER via Knowledge Retrieval, Disambiguation and Reflective AnalysisWenxuan Mu, Jinzhong Ning, Di Zhao, Yijia ZhangAAAI 2026 · 1 citation
- DiZiNER: Disagreement-guided Instruction Refinement via Simulating Pilot Annotation for Zero-shot Named Entity RecognitionSiun Kim, Hyung-Jin YoonACL 2026
