DeepEyesV2: Toward Agentic Multimodal Model
Jack Hong, Chenxiao Zhao, ChengLIn Zhu, Weiheng Lu, Guohai Xu, XingYu
Abstract
Agentic multimodal models should not only comprehend text and images, but also actively invoke external tools, such as code execution environments and web search, and integrate these operations into reasoning. In this work, we introduce DeepEyesV2 and explore how to build an agentic multimodal model from the perspectives of data construction, training methods, and model evaluation. We observe that direct reinforcement learning alone fails to induce robust tool-use behavior. This phenomenon motivates a two-stage training pipeline: a cold-start stage to establish tool-use patterns, and reinforcement learning stage to further refine tool invocation. We curate a diverse, moderately challenging training dataset, specifically including examples where tool use is beneficial. We further introduce RealX-Bench, a comprehensive benchmark designed to evaluate real-world multimodal reasoning, which inherently requires the integration of multiple capabilities, including perception, search, and reasoning. We evaluate DeepEyesV2 on RealX-Bench and other representative benchmarks, demonstrating its effectiveness across real-world understanding, mathematical reasoning, and search-intensive tasks. Moreover, DeepEyesV2 exhibits task-adaptive tool invocation, tending to use image operations for perception tasks and numerical computations for reasoning tasks. Reinforcement learning further enables complex tool combinations and allows model to selectively invoke tools based on context. We hope our study can provide guidance for community in developing agentic multimodal models.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 02b914c3-0e89-4d45-9497-c4a8f1d9d524Cited by top-tier papers7
- What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-ZoomMa Yan, Weiyu Zhang, Tianle Li, Du Linge et al.ICML 2026 · 5 citations
- EagleVision: A Dual-Stage Framework with BEV-grounding-based Chain-of-Thought for Spatial IntelligenceJiaxu Wan, Xu Wang, Mengwei Xie, Hang Zhang et al.CVPR 2026 · 3 citations
- From Where Things Are to What They Are For: Benchmarking Spatial–Functional Intelligence in Multimodal LLMsLe Zhang, Jihan Yang, Soundarya Krishnan, Jimit Majmudar et al.CVPR 2026 · 2 citations
- ACE-Router: Generalizing History-Aware Routing from MCP Tools to the Agent WebZhiyuan Yao, Zishan Xu, Yifu Guo, Zhiguang Han et al.ACL 2026 · 1 citation
- SAKE: Self-aware Knowledge Exploitation-Exploration for Grounded Multimodal Named Entity RecognitionJielong Tang, Xujie Yuan, Jiayang Liu, Jianxing Yu et al.KDD 2026 · 1 citation
Builds on17
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
- DAPO: An Open-Source LLM Reinforcement Learning System at ScaleQiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan et al.NeurIPS 2025 · 2,828 citations
- ReTool: Reinforcement Learning for Strategic Tool Use in LLMsJiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang et al.ICLR 2026 · 406 citations
- VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement LearningHaozhe Wang, Chao Qu, Zuming Huang, Wei Chu et al.NeurIPS 2025 · 356 citations
Related papers
- SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RLSiyi Chen, Mikaela Angelina Uy, Chan Hee Song, Faisal Ladhak et al.CVPR 2026 · 24 citations
- WebWatcher: Breaking New Frontiers of Vision-Language Deep Research AgentXinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang et al.ICLR 2026 · 79 citations
- Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic TasksTajamul Ashraf, Amal Saqib, Hanan Gani, Muhra AlMahri et al.ICLR 2026 · 15 citations
- ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual ReasoningShengyuan Ding, Xinyu Fang, Ziyu Liu, Yuhang Zang et al.CVPR 2026 · 11 citations
- MM-DeepResearch: A Simple and Effective Multimodal Agentic Search BaselineHuanjin Yao, Qixiang Yin, Min Yang, Ziwang Zhao et al.ICML 2026 · 14 citations
