Generation-Augmented Video Corpus Moment Retrieval
Mingjin Kuai, Qianyin Xiao, Juncheng Li, Jin Peng, Lizi Liao, Wei Ji
Abstract
Video Corpus Moment Retrieval (VCMR) requires models to efficiently retrieve and precisely locate specific moments relevant to natural language queries within a massive, untrimmed video corpus. However, existing discriminative approaches typically rely on shallow visual-textual feature matching mechanisms, which often struggle to capture fine-grained semantic differences. To address this limitation, we propose Video-GAR, a novel framework that reframes the conventional retrieval task from superficial matching to generative understanding, positing that the capability for query reconstruction evidences deep semantic comprehension. Specifically, Video-GAR orchestrates three synergistic components: To overcome the computational efficiency bottleneck, we construct a Bi-Mamba backbone that leverages the linear complexity of state-space models for efficient global context modeling. Building on these representations, we introduce a generation-augmented fusion module, in which a training-only decoder acts as a semantic regularizer to implicitly calibrate cross-modal attention without increasing inference overhead. Finally, to ensure fine-grained precision, we propose a boundary-aware localization strategy that integrates boundary modeling with categorical supervision. Experiments on two benchmark datasets demonstrate that Video-GAR significantly improves retrieval and localization accuracy while maintaining outstanding inference speed.
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Related papers
- Generalized Video Moment RetrievalYou Qin, Qilong Wu, Yicong Li, Wei Ji et al.ICLR 2025
- Modal-specific Pseudo Query Generation for Video Corpus Moment RetrievalMinjoon Jung, Seongho Choi, Joochan Kim, Jin-Hwa Kim et al.EMNLP 2022 · 11 citations
- Faster Video Moment Retrieval with Point-Level SupervisionXun Jiang, Zailei Zhou, Xing Xu, Yang Yang et al.ACM MM 2023 · 24 citations
- Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment LocalizationDaizong Liu, Xiaoye Qu, Xiao-Yang Liu, Jianfeng Dong et al.ACM MM 2020 · 115 citations
- Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language ModelsYifang Xu, Yunzhuo Sun, Benxiang Zhai, Ming Li et al.AAAI 2025 · 17 citations
