EMNLP Findings2025
Spotlighter: Revisiting Prompt Tuning from a Representative Mining View
Yutong Gao, Maoyuan Shao, Xinyang Huang, Chuang Zhu, Yu Weng, Xuan Liu, Lijuan Sun, Guoshun Nan
Abstract
CLIP's success has demonstrated that prompt tuning can achieve robust cross-modal semantic alignment for tasks ranging from open-domain recognition to fine-grained classification. However, redundant or weakly relevant feature components introduce noise and incur unnecessary computational costs. In this work, we propose Spotlighter, a lightweight token-selection framework that simultaneously enhances accuracy and efficiency in prompt tuning. Spotlighter evaluates each visual token's activation from both sample-wise and semantic-wise perspectives and retains only the top-scoring tokens for downstream prediction. A classspecific semantic memory bank of learned prototypes refines this selection, ensuring semantic representativeness and compensating for discarded features. To further prioritize informative signals, we introduce a two-level ranking mechanism that dynamically weights token-prototype interactions. Across 11 few-shot benchmarks, Spotlighter outperforms CLIP by up to 11.19% in harmonic mean accuracy and achieves up to 0.8K additional FPS, with only 21 extra parameters. These results establish Spotlighter as an effective and scalable baseline for prompt tuning. Code for our method will be available at https://github.com/greatest -gourmet/Spotlighter.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Align before Fuse: Vision and Language Representation Learning with Momentum Distillation cites Learning Transferable Visual Models From Natural Language Supervision
- Prompt-aligned Gradient for Prompt Tuning cites Learning Transferable Visual Models From Natural Language Supervision
- CAPT: Confusion-Aware Prompt Tuning for Reducing Vision-Language Misalignment cites Prompt-aligned Gradient for Prompt Tuning
- CAPT: Confusion-Aware Prompt Tuning for Reducing Vision-Language Misalignment cites Learning Transferable Visual Models From Natural Language Supervision
- Open-vocabulary Object Detection via Vision and Language Knowledge Distillation cites Learning Transferable Visual Models From Natural Language Supervision
- Learning Transferable Visual Models From Natural Language Supervision cites Language Models are Few-Shot Learners
Cited by top-tier papers1
Builds on20
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Align before Fuse: Vision and Language Representation Learning with Momentum DistillationJunnan Li, Ramprasaath R. Selvaraju, Akhilesh Gotmare, Shafiq R. Joty et al.NeurIPS 2021 · 2,985 citations
- Open-vocabulary Object Detection via Vision and Language Knowledge DistillationXiuye Gu, Tsung-Yi Lin, Weicheng Kuo, Yin CuiICLR 2022 · 1,274 citations
- Prompt-aligned Gradient for Prompt TuningBeier Zhu, Yulei Niu, Yucheng Han, Yue Wu et al.ICCV 2023 · 475 citations
Related papers
- Tuning Multi-mode Token-level Prompt Alignment across ModalitiesDongsheng Wang, Miaoge Li, Xinyang Liu, Mingsheng Xu et al.NeurIPS 2023 · 49 citations
- Compound Text-Guided Prompt Tuning via Image-Adaptive CuesHao Tan, Jun Li, Yizhuang Zhou, Jun Wan et al.AAAI 2024
- Visual and Semantic Prompt Collaboration for Generalized Zero-Shot LearningHuajie Jiang, Zhengxian Li, Xiaohan Yu, Yongli Hu et al.CVPR 2025
- Deeply Coupled Cross-Modal Prompt LearningXuejing Liu, Wei Tang, Jinghui Lu, Rui Zhao et al.ACL Findings 2023 · 17 citations
- Multi-modal Prototype Guided Few-shot Object DetectionChenbo Zhang, Bing Huangfu, Hongxu Ma, Jihong Guan et al.ACM MM 2025 · 3 citations
