VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation
Ming Dai, Sen Yang, Boqiang Duan, Boyuan Tong, Jiedong Zhuang, Wankou Yang, Jingdong Wang
摘要
Reasoning Video Object Segmentation (RVOS) demands a sophisticated integration of temporal dynamics, spatial details, and linguistic reasoning to achieve precise pixel-level localization. Existing methods are limited to reasoning over fixed initial inputs and lack the capacity to actively acquire further visual evidence, which is often essential for resolving complex references in long or intricate videos. To address this, we propose , the first multi-turn reinforcement learning framework for RVOS that emulates the human cognitive process. It employs a to capture fine-grained details by iteratively pinpointing critical intervals and keyframes. Additionally, to enable the policy to perceive segmentation quality beyond mere text probability of during the RL stage, we introduce , which integrates pixel-wise segmentation feedback directly into the token-level logits. Furthermore, we design a to hierarchically decompose the reasoning process into temporal, spatial, and linguistic dimensions, and construct , a specialized cold-start dataset featuring comprehensive reasoning trajectories. Extensive experiments demonstrate that VideoSEG-O3 achieves advanced performance across 8 mainstream RVOS benchmarks, particularly excelling in long-horizon and complex reasoning tasks.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper24
- Self-Chained Image-Language Model for Video Localization and Question AnsweringShoubin Yu, Jaemin Cho, Prateek Yadav, Mohit BansalNeurIPS 2023 · 被引用 281 次
- OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and UnderstandingTao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan 等NeurIPS 2024 · 被引用 186 次
- VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language TasksJiannan Wu, Muyan Zhong, Sen Xing, Zeqiang Lai 等NeurIPS 2024 · 被引用 179 次
- One Token to Seg Them All: Language Instructed Reasoning Segmentation in VideosZechen Bai, Tong He, Haiyang Mei, Pichao Wang 等NeurIPS 2024 · 被引用 147 次
- Language as Queries for Referring Video Object SegmentationJiannan Wu, Yi Jiang, Peize Sun, Zehuan Yuan 等CVPR 2022 · 被引用 143 次
相关 Paper
- VideoSeg-R1: Reasoning Video Object Segmentation via Reinforcement LearningZishan Xu, Yifu Guo, Yuquan Lu, Fengyu Yang 等AAAI 2026
- CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for VideosShiu-Hong Kao, Yu-Wing Tai, Chi-Keung TangICLR 2026 · 被引用 8 次
- Reinforcing Video Reasoning Segmentation to Think Before It SegmentsSitong Gong, Yunzhi Zhuge, Lu Zhang, Jiazuo Yu 等CVPR 2026 · 被引用 16 次
- Training-Free Spatio-temporal Decoupled Reasoning Video Segmentation with Adaptive Object MemoryZhengtong Zhu, Jiaqing Fan, Zhixuan Liu, Fanzhang LiAAAI 2026 · 被引用 1 次
- VIRST: Video-Instructed Reasoning Assistant for SpatioTemporal SegmentationJihwan Hong, Jaeyoung DoCVPR 2026 · 被引用 2 次
