ACL Findings2023
Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding
Erica Kido Shimomoto, Edison Marrese-Taylor, Hiroya Takamura, Ichiro Kobayashi, Hideki Nakayama, Yusuke Miyao
Abstract
This paper explores the task of Temporal Video Grounding (TVG) where, given an untrimmed video and a natural language sentence query, the goal is to recognize and determine temporal boundaries of action instances in the video described by the query. Recent works tackled this task by improving query inputs with large pre-trained language models (PLM) at the cost of more expensive training. However, the effects of this integration are unclear, as these works also propose improvements in the visual inputs. Therefore, this paper studies the effects of PLMs in TVG and assesses the applicability of parameter-efficient training with NLP adapters. We couple popular PLMs with a selection of existing approaches and test different adapters to reduce the impact of the additional parameters. Our results on three challenging datasets show that, without changing the visual inputs, TVG models greatly benefited from the PLM integration and fine-tuning, stressing the importance of sentence query representation in this task. Furthermore, NLP adapters were an effective alternative to full fine-tuning, even though they were not tailored to our task, allowing PLM integration in larger TVG models and delivering results comparable to SOTA models. Finally, our results shed light on which adapters work best in different scenarios.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- LoRA: Low-Rank Adaptation of Large Language Models cites Compacter: Efficient Low-Rank Hypercomplex Adapter Layers
- Multi-to-Single: Reducing Multimodal Dependency in Emotion Recognition Through Contrastive Learning cites Learning Transferable Visual Models From Natural Language Supervision
- Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection cites Learning Transferable Visual Models From Natural Language Supervision
- eP-ALM: Efficient Perceptual Augmentation of Language Models cites LoRA: Low-Rank Adaptation of Large Language Models
- eP-ALM: Efficient Perceptual Augmentation of Language Models cites Compacter: Efficient Low-Rank Hypercomplex Adapter Layers
- eP-ALM: Efficient Perceptual Augmentation of Language Models cites Multimodal Few-Shot Learning with Frozen Language Models
- One Last Attention for Your Vision-Language Model cites Learning Transferable Visual Models From Natural Language Supervision
- Multimodal Few-Shot Learning with Frozen Language Models cites Learning Transferable Visual Models From Natural Language Supervision
Cited by top-tier papers4
- Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight DetectionYicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma et al.CVPR 2024 · 43 citations
- eP-ALM: Efficient Perceptual Augmentation of Language ModelsMustafa Shukor, Corentin Dancette, Matthieu CordICCV 2023 · 36 citations
- Multi-to-Single: Reducing Multimodal Dependency in Emotion Recognition Through Contrastive LearningYan-Kai Liu, Jinyu Cai, Bao-Liang Lu, Wei-Long ZhengAAAI 2025 · 6 citations
- One Last Attention for Your Vision-Language ModelLiang Chen, Ghazi Shazan Ahmad, Tianjun Yao, Lingqiao Liu et al.ICCV 2025 · 1 citation
Builds on25
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- Multimodal Few-Shot Learning with Frozen Language ModelsMaria Tsimpoukelli, Jacob Menick, Serkan Cabi, S. M. Ali Eslami et al.NeurIPS 2021 · 1,020 citations
- Compacter: Efficient Low-Rank Hypercomplex Adapter LayersRabeeh Karimi Mahabadi, James Henderson, Sebastian RuderNeurIPS 2021 · 700 citations
Related papers
- Enrich and Detect: Video Temporal Grounding With Multimodal LlmsShraman Pramanick, Effrosyni Mavroudi, Yale Song, Rama Chellappa et al.ICCV 2025 · 4 citations
- Time-R1: Post-Training Large Vision Language Model for Temporal Video GroundingYe Wang, Ziheng Wang, Boshen Xu, Yang Du et al.NeurIPS 2025 · 143 citations
- Universal Video Temporal Grounding with Generative Multi-modal Large Language ModelsZeqian Li, Shangzhe Di, Zhonghua Zhai, Weilin Huang et al.NeurIPS 2025 · 30 citations
- ProTéGé: Untrimmed Pretraining for Video Temporal Grounding by Video Temporal GroundingLan Wang, Gaurav Mittal, Sandra Sajeev, Ye Yu et al.CVPR 2023
- Vid-Group: Temporal Video Grounding Pretraining from Unlabeled Videos in the WildPeijun Bao, Chenqi Kong, Siyuan Yang, Zihao Shao et al.ICCV 2025 · 3 citations
