Lune

EMNLP Findings2025

Spotlighter: Revisiting Prompt Tuning from a Representative Mining View

Yutong Gao, Maoyuan Shao, Xinyang Huang, Chuang Zhu, Yu Weng, Xuan Liu, Lijuan Sun, Guoshun Nan

Abstract

CLIP's success has demonstrated that prompt tuning can achieve robust cross-modal semantic alignment for tasks ranging from open-domain recognition to fine-grained classification. However, redundant or weakly relevant feature components introduce noise and incur unnecessary computational costs. In this work, we propose Spotlighter, a lightweight token-selection framework that simultaneously enhances accuracy and efficiency in prompt tuning. Spotlighter evaluates each visual token's activation from both sample-wise and semantic-wise perspectives and retains only the top-scoring tokens for downstream prediction. A classspecific semantic memory bank of learned prototypes refines this selection, ensuring semantic representativeness and compensating for discarded features. To further prioritize informative signals, we introduce a two-level ranking mechanism that dynamically weights token-prototype interactions. Across 11 few-shot benchmarks, Spotlighter outperforms CLIP by up to 11.19% in harmonic mean accuracy and achieves up to 0.8K additional FPS, with only 21 extra parameters. These results establish Spotlighter as an effective and scalable baseline for prompt tuning. Code for our method will be available at https://github.com/greatest -gourmet/Spotlighter.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Align before Fuse: Vision and Language Representation Learning with Momentum Distillation cites Learning Transferable Visual Models From Natural Language Supervision
  • Prompt-aligned Gradient for Prompt Tuning cites Learning Transferable Visual Models From Natural Language Supervision
  • CAPT: Confusion-Aware Prompt Tuning for Reducing Vision-Language Misalignment cites Prompt-aligned Gradient for Prompt Tuning
  • CAPT: Confusion-Aware Prompt Tuning for Reducing Vision-Language Misalignment cites Learning Transferable Visual Models From Natural Language Supervision
  • Open-vocabulary Object Detection via Vision and Language Knowledge Distillation cites Learning Transferable Visual Models From Natural Language Supervision
  • Learning Transferable Visual Models From Natural Language Supervision cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. CAPT: Confusion-Aware Prompt Tuning for Reducing Vision-Language MisalignmentMaoyuan Shao, Yutong Gao, Xinyang Huang, Lijuan Sun et al.CVPR 2026 · 1 citation

Builds on20

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  3. Align before Fuse: Vision and Language Representation Learning with Momentum DistillationJunnan Li, Ramprasaath R. Selvaraju, Akhilesh Gotmare, Shafiq R. Joty et al.NeurIPS 2021 · 2,985 citations
  4. Open-vocabulary Object Detection via Vision and Language Knowledge DistillationXiuye Gu, Tsung-Yi Lin, Weicheng Kuo, Yin CuiICLR 2022 · 1,274 citations
  5. Prompt-aligned Gradient for Prompt TuningBeier Zhu, Yulei Niu, Yucheng Han, Yue Wu et al.ICCV 2023 · 475 citations

Related papers

  1. Tuning Multi-mode Token-level Prompt Alignment across ModalitiesDongsheng Wang, Miaoge Li, Xinyang Liu, Mingsheng Xu et al.NeurIPS 2023 · 49 citations
  2. Compound Text-Guided Prompt Tuning via Image-Adaptive CuesHao Tan, Jun Li, Yizhuang Zhou, Jun Wan et al.AAAI 2024
  3. Visual and Semantic Prompt Collaboration for Generalized Zero-Shot LearningHuajie Jiang, Zhengxian Li, Xiaohan Yu, Yongli Hu et al.CVPR 2025
  4. Deeply Coupled Cross-Modal Prompt LearningXuejing Liu, Wei Tang, Jinghui Lu, Rui Zhao et al.ACL Findings 2023 · 17 citations
  5. Multi-modal Prototype Guided Few-shot Object DetectionChenbo Zhang, Bing Huangfu, Hongxu Ma, Jihong Guan et al.ACM MM 2025 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines