Lune

SIGIR2021Top-tier venue

Improving Video Retrieval by Adaptive Margin

Feng He, Qi Wang, Zhifan Feng, Wenbin Jiang, Yajuan Lü, Yong Zhu, Xiao Tan

Abstract

Video retrieval is becoming increasingly important owing to the rapid emergence of videos on the Internet. The dominant paradigm for video retrieval learns video-text representations by pushing the distance between the similarity of positive pairs and that of negative pairs apart from a fixed margin. However, negative pairs used for training are sampled randomly, which indicates that the semantics between negative pairs may be related or even equivalent, while most methods still enforce dissimilar representations to decrease their similarity. This phenomenon leads to inaccurate supervision and poor performance in learning video-text representations.

While most video retrieval methods overlook that phenomenon, we propose an adaptive margin changed with the distance between positive and negative pairs to solve the aforementioned issue. First, we design the calculation framework of the adaptive margin, including the method of distance measurement and the function between the distance and the margin. Then, we explore a novel implementation called "Cross-Modal Generalized Self-Distillation" (CMGSD), which can be built on the top of most video retrieval models with few modifications. Notably, CMGSD adds few computational overheads at train time and adds no computational overhead at test time. Experimental results on three widely used datasets demonstrate that the proposed method can yield significantly better performance than the corresponding backbone model, and it outperforms state-of-the-art methods by a large margin.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

6 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LGDN: Language-Guided Denoising Network for Video-Language Modeling cites HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
  • Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings cites HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
  • MUSE: Mamba Is Efficient Multi-scale Learner for Text-video Retrieval cites X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval
  • X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval cites Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval
  • Continual Text-to-Video Retrieval with Frame Fusion and Task-Aware Routing cites HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
  • Continual Text-to-Video Retrieval with Frame Fusion and Task-Aware Routing cites CenterCLIP: Token Clustering for Efficient Text-Video Retrieval
  • Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval cites HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
  • Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval cites Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings
  • CenterCLIP: Token Clustering for Efficient Text-Video Retrieval cites HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers6

  1. X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text RetrievalYiwei Ma, Guohai Xu, Xiaoshuai Sun, Ming Yan et al.ACM MM 2022 · 314 citations
  2. CenterCLIP: Token Clustering for Efficient Text-Video RetrievalShuai Zhao, Linchao Zhu, Xiaohan Wang, Yi YangSIGIR 2022 · 150 citations
  3. LGDN: Language-Guided Denoising Network for Video-Language ModelingHaoyu Lu, Mingyu Ding, Nanyi Fei, Yuqi Huo et al.NeurIPS 2022 · 20 citations
  4. MUSE: Mamba Is Efficient Multi-scale Learner for Text-video RetrievalHaoran Tang, Meng Cao, Jinfa Huang, Ruyang Liu et al.AAAI 2025 · 13 citations
  5. Continual Text-to-Video Retrieval with Frame Fusion and Task-Aware RoutingZecheng Zhao, Zhi Chen, Zi Huang, Shazia Sadiq et al.SIGIR 2025 · 6 citations

Builds on7

  1. HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsAntoine Miech, Dimitri Zhukov, Jean-Baptiste Alayrac, Makarand Tapaswi et al.ICCV 2019 · 1,437 citations
  2. Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self DistillationLinfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen et al.ICCV 2019 · 1,069 citations
  3. Self-Distillation Amplifies Regularization in Hilbert SpaceHossein Mobahi, Mehrdad Farajtabar, Peter L. BartlettNeurIPS 2020 · 298 citations
  4. Fine-Grained Action Retrieval Through Multiple Parts-of-Speech EmbeddingsMichael Wray, Gabriela Csurka, Diane Larlus, Dima DamenICCV 2019 · 185 citations
  5. Tree-Augmented Cross-Modal Encoding for Complex-Query Video RetrievalXun Yang, Jianfeng Dong, Yixin Cao, Xun Wang et al.SIGIR 2020 · 131 citations

Related papers

  1. Progressive Semantic Matching for Video-Text RetrievalHongying Liu, Ruyi Luo, Fanhua Shang, Mantang Niu et al.ACM MM 2021 · 20 citations
  2. Dual Alignment Unsupervised Domain Adaptation for Video-Text RetrievalXiaoshuai Hao, Wanqian Zhang, Dayan Wu, Fei Zhu et al.CVPR 2023
  3. Learning Semantics-Grounded Vocabulary Representation for Video-Text RetrievalYaya Shi, Haowei Liu, Haiyang Xu, Zongyang Ma et al.ACM MM 2023 · 3 citations
  4. Text-Adaptive Multiple Visual Prototype Matching for Video-Text RetrievalChengzhi Lin, Ancong Wu, Junwei Liang, Jun Zhang et al.NeurIPS 2022 · 52 citations
  5. Learning Audio-guided Video Representation with Gated Attention for Video-Text RetrievalBoseung Jeong, Jicheol Park, Sungyeon Kim, Suha KwakCVPR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines