Activation Matters: Test-time Activated Negative Labels for OOD Detection with Vision-Language Models
Yabin Zhang, Maya Varma, Yunhe Gao, Jean-Benoit Delbrouck, Jiaming Liu, Chong Wang, Curtis Langlotz
Abstract
Out-of-distribution (OOD) detection aims to identify samples that deviate from in-distribution (ID). One popular pipeline addresses this by introducing negative labels distant from ID classes and detecting OOD based on their distance to these labels. However, such labels may present poor activation on OOD samples, failing to capture the OOD characteristics. To address this, we propose Test-time Activated Negative Labels (TANL) by dynamically evaluating activation levels across the corpus dataset and mining candidate labels with high activation responses during the testing process. Specifically, TANL identifies high-confidence test images online and accumulates their assignment probabilities over the corpus to construct a label activation metric. Such a metric leverages historical test samples to adaptively align with the test distribution, enabling the selection of distribution-adaptive activated negative labels. By further exploring the activation information within the current testing batch, we introduce a more fine-grained, batch-adaptive variant. To fully utilize label activation knowledge, we propose an activation-aware score function that emphasizes negative labels with stronger activations, boosting performance and enhancing its robustness to the label number. Our TANL is training-free, test-efficient, and grounded in theoretical justification. Experiments on diverse backbones and wide task settings validate its effectiveness. Notably, on the large-scale ImageNet benchmark, TANL significantly reduces the FPR95 from 17.5% to 9.8%. Codes are available at YBZh/OpenOOD-VLM.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision cites The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision cites Learning Transferable Visual Models From Natural Language Supervision
- The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization cites AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
- Learning Transferable Visual Models From Natural Language Supervision cites The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
Builds on54
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text SupervisionChao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen et al.ICML 2021 · 5,401 citations
- The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution GeneralizationDan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath et al.ICCV 2021 · 2,294 citations
- Energy-based Out-of-distribution DetectionWeitang Liu, Xiaoyun Wang, John D. Owens, Yixuan LiNeurIPS 2020 · 2,213 citations
- AugMix: A Simple Data Processing Method to Improve Robustness and UncertaintyDan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph et al.ICLR 2020 · 1,572 citations
Related papers
- AdaNeg: Adaptive Negative Proxy Guided OOD Detection with Vision-Language ModelsYabin Zhang, Lei ZhangNeurIPS 2024 · 30 citations
- Test-Time Linear Out-of-Distribution DetectionKe Fan, Tong Liu, Xingyu Qiu, Yikai Wang et al.CVPR 2024
- AdaSCALE: Adaptive Scaling for OOD DetectionSudarshan RegmiICML 2026 · 9 citations
- TTL: Test-time Textual Learning for OOD Detection with Pretrained Vision-Language ModelsJinlun Ye, Jiang Liao, Runhe Lai, Xinhua Lu et al.CVPR 2026 · 2 citations
- ActMAD: Activation Matching to Align Distributions for Test-Time-TrainingMuhammad Jehanzeb Mirza, Pol Jané-Soneira, Wei Lin, Mateusz Kozinski et al.CVPR 2023
