ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training
Dunwei Tu, Hongyan Hao, Hansi Yang, Yihao Chen, Yu Yang, Yueqing Sun, Xingchen Liu, Furao Shen, Qi GU, Hui Su, Xunliang Cai
Abstract
Training generalist agents capable of adapting to diverse scenarios requires interactive environments for self-exploration. However, interactive environments remain critically scarce, and existing synthesis methods suffer from significant limitations regarding environmental diversity and scalability. To address these challenges, we introduce ScaleEnv, a framework that constructs fully interactive environments and verifiable tasks entirely from scratch. Specifically, ScaleEnv ensures environment reliability through procedural testing, and guarantees task completeness and solvability via tool dependency graph expansion and executable action verification. By enabling agents to learn through exploration within ScaleEnv, we demonstrate significant performance improvements on unseen, multi-turn tool-use benchmarks such as -Bench and VitaBench, highlighting strong generalization capabilities. Furthermore, we investigate the relationship between increasing number of domains and model generalization performance, providing empirical evidence that scaling environmental diversity is critical for robust agent learning.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on4
- Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
- Feedback Loops With Language Models Drive In-Context Reward HackingAlexander Pan, Erik Jones, Meena Jagadeesan, Jacob SteinhardtICML 2024 · 67 citations
- Scaling Agent Learning via Experience SynthesisZhaorun Chen, Zhuokai Zhao, Kai Zhang, Bo Liu et al.ICLR 2026 · 37 citations
- ReAct: Synergizing Reasoning and Acting in Language ModelsShunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du et al.ICLR 2023
Related papers
- Safe and Scalable Web Agent Learning via Recreated WebsitesHyungjoo Chae, Jungsoo Park, Alan RitterICML 2026
- Generalizable End-to-End Tool-Use RL with Synthetic CodeGymWeihua Du, Hailei Gong, Zhan Ling, Kang Liu et al.ICLR 2026 · 13 citations
- Scaling Agentic Capabilities via Grounded Interaction SynthesisWenhang Shi, Jinhao Dong, Yiren Chen, Zhe Zhao et al.KDD 2026 · 1 citation
- Large-Scale Terminal Agentic Trajectory Generation from Dockerized EnvironmentsSiwei Wu, Yizhi Li, Yuyang Song, Wei Zhang et al.ICML 2026 · 16 citations
- Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement LearningZhaoyang Wang, Canwen Xu, Boyi Liu, Yite Wang et al.ICML 2026 · 25 citations
