AgentTuning: Enabling Generalized Agent Abilities for LLMs

Open large language models (LLMs) has thus far inferior to commercial LLMs when acting as agents to tackle complex tasks. These agent tasks employ LLMs as the central controller responsible for planning, memorization, and tool utilization. To date, there is lack of research focusing on improving the agent capabilities of LLMs themselves. In this work, we present AgentTuning, a simple and general method to enhance the agent abilities of LLMs while maintaining their general LLM capabilities. We construct AgentInstruct, a lightweight instruction-tuning dataset containing high-quality interaction trajectories. We employ a hybrid instruction-tuning strategy by combining AgentInstruct with open-source instructions from general domains. AgentTuning is used to instruction-tune the Llama 2 series, resulting in AgentLM. Evaluations show that AgentTuning enables LLMs' agent capabilities without compromising general abilities. The AgentLM-70B is comparable to GPT-3.5turbo on unseen agent tasks, demonstrating generalized agent capabilities. We open source the AgentInstruct dataset and AgentLM-7B, 13B, and 70B models at https://anonymous. 4open.science/r/AgentTuning.