LONGAGENT: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration

Large language models (LLMs) have achieved tremendous success in understanding language and processing text. However, questionanswering (QA) on lengthy documents faces challenges of resource constraints and a high propensity for errors, even for the most advanced models such as GPT-4 and Claude2. In this paper, we introduce LONGAGENT, a multi-agent collaboration method that enables efficient and effective QA over 128k-tokenlong documents. LONGAGENT adopts a divideand-conquer strategy, breaking down lengthy documents into shorter, more manageable text chunks. A leader agent comprehends the user's query and organizes the member agents to read their assigned chunks, reasoning a final answer through multiple rounds of discussion. Due to members' hallucinations, it's difficult to guarantee that every response provided by each member is accurate. To address this, we develop an inter-member communication mechanism that facilitates information sharing, allowing for the detection and mitigation of hallucinatory responses. Experimental results show that a LLaMA-2 7B driven by LONGAGENT can effectively support QA over 128k-token documents, achieving 16.42% and 1.63% accuracy gains over GPT-4 on singlehop and multi-hop QA settings, respectively.