DeepEyesV2: Toward Agentic Multimodal Model
Jack Hong, Chenxiao Zhao, ChengLIn Zhu, Weiheng Lu, Guohai Xu, XingYu
摘要
Agentic multimodal models should not only comprehend text and images, but also actively invoke external tools, such as code execution environments and web search, and integrate these operations into reasoning. In this work, we introduce DeepEyesV2 and explore how to build an agentic multimodal model from the perspectives of data construction, training methods, and model evaluation. We observe that direct reinforcement learning alone fails to induce robust tool-use behavior. This phenomenon motivates a two-stage training pipeline: a cold-start stage to establish tool-use patterns, and reinforcement learning stage to further refine tool invocation. We curate a diverse, moderately challenging training dataset, specifically including examples where tool use is beneficial. We further introduce RealX-Bench, a comprehensive benchmark designed to evaluate real-world multimodal reasoning, which inherently requires the integration of multiple capabilities, including perception, search, and reasoning. We evaluate DeepEyesV2 on RealX-Bench and other representative benchmarks, demonstrating its effectiveness across real-world understanding, mathematical reasoning, and search-intensive tasks. Moreover, DeepEyesV2 exhibits task-adaptive tool invocation, tending to use image operations for perception tasks and numerical computations for reasoning tasks. Reinforcement learning further enables complex tool combinations and allows model to selectively invoke tools based on context. We hope our study can provide guidance for community in developing agentic multimodal models.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper7
- What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-ZoomMa Yan, Weiyu Zhang, Tianle Li, Du Linge 等ICML 2026 · 被引用 5 次
- EagleVision: A Dual-Stage Framework with BEV-grounding-based Chain-of-Thought for Spatial IntelligenceJiaxu Wan, Xu Wang, Mengwei Xie, Hang Zhang 等CVPR 2026 · 被引用 3 次
- From Where Things Are to What They Are For: Benchmarking Spatial–Functional Intelligence in Multimodal LLMsLe Zhang, Jihan Yang, Soundarya Krishnan, Jimit Majmudar 等CVPR 2026 · 被引用 2 次
- ACE-Router: Generalizing History-Aware Routing from MCP Tools to the Agent WebZhiyuan Yao, Zishan Xu, Yifu Guo, Zhiguang Han 等ACL 2026 · 被引用 1 次
- SAKE: Self-aware Knowledge Exploitation-Exploration for Grounded Multimodal Named Entity RecognitionJielong Tang, Xujie Yuan, Jiayang Liu, Jianxing Yu 等KDD 2026 · 被引用 1 次
它引用的顶会 Paper17
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 被引用 11,349 次
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 被引用 7,873 次
- DAPO: An Open-Source LLM Reinforcement Learning System at ScaleQiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan 等NeurIPS 2025 · 被引用 2,828 次
- ReTool: Reinforcement Learning for Strategic Tool Use in LLMsJiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang 等ICLR 2026 · 被引用 406 次
- VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement LearningHaozhe Wang, Chao Qu, Zuming Huang, Wei Chu 等NeurIPS 2025 · 被引用 356 次
相关 Paper
- SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RLSiyi Chen, Mikaela Angelina Uy, Chan Hee Song, Faisal Ladhak 等CVPR 2026 · 被引用 24 次
- WebWatcher: Breaking New Frontiers of Vision-Language Deep Research AgentXinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang 等ICLR 2026 · 被引用 79 次
- Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic TasksTajamul Ashraf, Amal Saqib, Hanan Gani, Muhra AlMahri 等ICLR 2026 · 被引用 15 次
- ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual ReasoningShengyuan Ding, Xinyu Fang, Ziyu Liu, Yuhang Zang 等CVPR 2026 · 被引用 11 次
- MM-DeepResearch: A Simple and Effective Multimodal Agentic Search BaselineHuanjin Yao, Qixiang Yin, Min Yang, Ziwang Zhao 等ICML 2026 · 被引用 14 次
