Dual-Phase Visual-Language Pretraining and Adaptation for Long-Tailed Multi-Label Recognition
Yongcheng Li, Xuekuan Wang, Zhifei Zhang, Cairong Zhao
Abstract
Long-Tailed Multi-Label Recognition (LTML) is a critical yet challenging task due to two core issues: the severe scarcity of training samples for rare "tail" classes, and the complex co-occurrence patterns among labels that often lead to biased models. To address this, we propose DP-VLPA, a novel Dual-Phase Visual-Language Pretraining and Adaptation framework. In the first phase, our Structured Tail-Aware Generation (STAG) module employs a Large Language Model (LLM) to create detailed descriptions that explicitly emphasize tail classes and their contextual relationships, providing a strong and less-biased feature foundation. In the second adaptation phase, we ensure this knowledge is applied effectively. A Dynamic Query Reweighting (DQR) mechanism forces the model to attend to crucial tail-class evidence. Simultaneously, a Co-occurrence-Aware (COA) loss explicitly teaches the model the statistical dependencies between labels, correcting for co-occurrence biases. Extensive experiments on VOC-LT and COCO-LT datasets demonstrate state-of-the-art performance, achieving mAP scores of 90.72% and 74.42% respectively - surpassing previous best methods by 2.84% and 8.23%.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext b7f057de-4893-401e-bb2e-3bfa3157e82aBuilds on16
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 citations
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text SupervisionChao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen et al.ICML 2021 · 5,401 citations
- ViLT: Vision-and-Language Transformer Without Convolution or Region SupervisionWonjae Kim, Bokyung Son, Ildoo KimICML 2021 · 2,258 citations
Related papers
- DART: Dual Adaptive Refinement Transfer for Open-Vocabulary Multi-Label RecognitionHaijing Liu, Tao Pu, Hefeng Wu, Keze Wang et al.ACM MM 2025
- Long-Tailed Multi-Label Visual Recognition by Collaborative Training on Uniform and Re-Balanced SamplingsHao Guo, Song WangCVPR 2021
- From Head to Tail: Towards Balanced Representation in Large Vision-Language Models through Adaptive Data CalibrationMingyang Song, Xiaoye Qu, Jiawei Zhou, Yu ChengCVPR 2025
- Long-tailed Object Detection Pretraining: Dynamic Rebalancing Contrastive Learning with Dual ReconstructionChen-Long Duan, Yong Li, Xiu-Shen Wei, Lin ZhaoNeurIPS 2024 · 9 citations
- Adaptive Token Refinement in Long-Tailed Large Vision-Language Models Fine-TuningWenjun Miao, Mingda Li, Yanchao Hao, Zheng WeiICML 2026
