AlignedNorm: Prompting Vision–Language Models via Coupled Prompt Field
Qi Ma, Chen-Yang Wang, Dehong Gao, Deng-Ping Fan
摘要
Prompt learning for vision-language models (VLMs) primarily follows end-to-end or decoupled routes to balance base and new task performance, but suffers a fundamental bottleneck: sample-wise optimization within task-specific feature spaces traps models in local optima, hindering global optimality. To address this, we identify a key insight that VLMs can be prompted within a Coupled Prompt Field -a shared space where base and new tasks are mutually constrained -and present AlignedNorm, which enforces the field coupling. By dynamically aligning the norms of prompts to VLMs' native scale, our method enables joint optimization of both tasks. Without complex designs, our method matches leading decoupled approaches on 15 datasets across 4 experimental settings, offering both a new perspective and a practical solution to the local-optimum dilemma in prompt learning. Code is available at https://github.com/QByteM/AlignedNorm.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper48
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text SupervisionChao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen 等ICML 2021 · 被引用 5,401 次
- Sigmoid Loss for Language Image Pre-TrainingXiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, Lucas BeyerICCV 2023 · 被引用 2,932 次
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the HypersphereTongzhou Wang, Phillip IsolaICML 2020 · 被引用 2,360 次
- The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution GeneralizationDan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath 等ICCV 2021 · 被引用 2,294 次
相关 Paper
- Enhancing Domain Adaptation through Prompt Gradient AlignmentViet Hoang Phan, Tung Lam Tran, Quyen Tran, Trung LeNeurIPS 2024 · 被引用 18 次
- MaPLe: Multi-modal Prompt LearningMuhammad Uzair Khattak, Hanoona Abdul Rasheed, Muhammad Maaz, Salman H. Khan 等CVPR 2023
- Advancing Prompt Learning through an External LayerFangming Cui, Xun Yang, Chao Wu, Liang Xiao 等ACM MM 2024 · 被引用 3 次
- Joint Scheduling of Causal Prompts and Tasks for Multi-Task LearningChaoyang Li, Jianyang Qin, Jinhao Cui, Zeyu Liu 等CVPR 2025
- Distribution-Aware Prompt Tuning for Vision-Language ModelsEulrang Cho, Jooyeon Kim, Hyunwoo J. KimICCV 2023 · 被引用 54 次
