CVPR Findings2026
MedSAD-CLIP: Supervised CLIP with Token-Patch Cross-Attention for Medical Anomaly Detection and Segmentation
Thuy Truong Tran, Minh Kha Do, Phuc Nguyen Duy, Min Hun Lee
Abstract
Medical anomaly detection (MAD) and segmentation play a critical role in assisting clinical diagnosis by identifying abnormal regions in medical images and localizing pathological regions. Recent CLIP-based studies are promising for anomaly detection in zero-/few-shot settings, and typically rely on global representations and weak supervision, often producing coarse localization and limited segmentation quality. In this work, we study supervised adaptation of CLIP for MAD under a realistic clinical setting where a limited yet meaningful amount of labeled abnormal data is available. Our model MedSAD-CLIP leverages fine-grained text-visual cues via the Token-Patch Cross-Attention (TPCA) to improve lesion localization while preserving the generalization capability of CLIP representations. Lightweight image adapters and learnable prompt tokens efficiently adapt the pretrained CLIP encoder to the medical domain while preserving its rich semantic alignment. Furthermore, a Margin-based image-text Contrastive Loss is designed to enhance global feature discrimination between normal and abnormal representations. Extensive experiments on four diverse benchmarks-Brain, Retina, Lung, and Breast datasets-demonstrate the effectiveness of our approach, achieving superior performance in both pixel-level segmentation and image-level classification over state-of-the-art methods. Our results highlight the potential of supervised CLIP adaptation as a unified and scalable paradigm for medical anomaly understanding. Code will be made available at https://github.com/ thuy4tbn99/MedSAD-CLIP
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision cites Learning Transferable Visual Models From Natural Language Supervision
- AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection cites Catching Both Gray and Black Swans: Open-set Supervised Anomaly Detection
- AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection cites CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
- AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection cites Learning Transferable Visual Models From Natural Language Supervision
Builds on14
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text SupervisionChao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen et al.ICML 2021 · 5,401 citations
- AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly DetectionQihang Zhou, Guansong Pang, Yu Tian, Shibo He et al.ICLR 2024 · 380 citations
- CLIP-Driven Universal Model for Organ Segmentation and Tumor DetectionJie Liu, Yixiao Zhang, Jieneng Chen, Junfei Xiao et al.ICCV 2023 · 336 citations
- Catching Both Gray and Black Swans: Open-set Supervised Anomaly DetectionChoubo Ding, Guansong Pang, Chunhua ShenCVPR 2022 · 163 citations
Related papers
- AF-CLIP: Zero-Shot Anomaly Detection via Anomaly-Focused CLIP AdaptationQingqing Fang, Wenxi Lv, Qinliang SuACM MM 2025 · 17 citations
- Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical ImagesChaoqin Huang, Aofan Jiang, Jinghao Feng, Ya Zhang et al.CVPR 2024
- AdaptCLIP: Adapting CLIP for Universal Visual Anomaly DetectionBin-Bin Gao, Yue Zhou, Jiangtao Yan, Yuezhi Cai et al.AAAI 2026 · 21 citations
- Aligning and Prompting Anything for Zero-Shot Generalized Anomaly DetectionJitao Ma, Weiying Xie, Hangyu Ye, Daixun Li et al.AAAI 2025 · 3 citations
- SimCLIP: Refining Image-Text Alignment with Simple Prompts for Zero-/Few-shot Anomaly DetectionChenghao Deng, Haote Xu, Xiaolu Chen, Haodi Xu et al.ACM MM 2024 · 10 citations
