Towards Training-free Anomaly Detection with Vision and Language Foundation Models
Jinjin Zhang, Guodong Wang, Yizhou Jin, Di Huang
Abstract
Anomaly detection is valuable for real-world applications, such as industrial quality inspection. However, most approaches focus on detecting local structural anomalies while neglecting compositional anomalies incorporating logical constraints. In this paper, we introduce LogSAD, a novel multi-modal framework that requires no training for both Logical and Structural Anomaly Detection. First, we propose a match-of-thought architecture that employs advanced large multi-modal models (i.e. GPT-4V) to generate matching proposals, formulating interests and compositional rules of thought for anomaly detection. Second, we elaborate on multi-granularity anomaly detection, consisting of patch tokens, sets of interests, and composition matching with vision and language foundation models. Subsequently, we present a calibration module to align anomaly scores from different detectors, followed by integration strategies for the final decision. Consequently, our approach addresses both logical and structural anomaly detection within a unified framework and achieves state-of-the-art results without the need for training, even when compared to supervised approaches, highlighting its robustness and effectiveness. Code is available at https://github.com/zhang0jhon/LogSAD .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
8 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Foundation Visual Encoders Are Secretly Few-Shot Anomaly Detectors cites Learning Transferable Visual Models From Natural Language Supervision
- Segment Anything cites Learning Transferable Visual Models From Natural Language Supervision
- Defect Cue-Preserved Structural Feature Refinement for Few-Shot Anomaly Detection cites Learning Transferable Visual Models From Natural Language Supervision
- Visual Instruction Tuning cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
- Seeing Clearly, Reasoning Confidently: Plug-and-Play Remedies for Vision Language Model Blindness cites Segment Anything
- Seeing Clearly, Reasoning Confidently: Plug-and-Play Remedies for Vision Language Model Blindness cites Visual Instruction Tuning
- Seeing Clearly, Reasoning Confidently: Plug-and-Play Remedies for Vision Language Model Blindness cites Learning Transferable Visual Models From Natural Language Supervision
- Reasoning-Driven Anomaly Detection and Localization with Image-Level Supervision cites Segment Anything
- Reasoning-Driven Anomaly Detection and Localization with Image-Level Supervision cites Learning Transferable Visual Models From Natural Language Supervision
Cited by top-tier papers8
- Foundation Visual Encoders Are Secretly Few-Shot Anomaly DetectorsGuangyao Zhai, Yue Zhou, Xinyan Deng, Lars Heckler-Kram et al.ICLR 2026 · 9 citations
- Reasoning-Driven Anomaly Detection and Localization with Image-Level SupervisionYizhou Jin, Yuezhu Feng, Jinjin Zhang, Peng Wang et al.CVPR 2026 · 4 citations
- AnomalyMoE: Towards a Language-free Generalist Model for Unified Visual Anomaly DetectionZhaopeng Gu, Bingke Zhu, Guibo Zhu, Yingying Chen et al.AAAI 2026 · 3 citations
- Seeing Clearly, Reasoning Confidently: Plug-and-Play Remedies for Vision Language Model BlindnessXin Hu, Haomiao Ni, Yunbei Zhang, Jihun Hamm et al.CVPR 2026 · 1 citation
- Defect Cue-Preserved Structural Feature Refinement for Few-Shot Anomaly DetectionLe Jiang, Yan Huang, Zhen Xu, Yong Xu et al.CVPR 2026
Builds on22
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao et al.ICCV 2023 · 13,211 citations
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
Related papers
- MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly DetectionXi Jiang, Jian Li, Hanqiu Deng, Yong Liu et al.ICLR 2025 · 3 citations
- AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language ModelsZhaopeng Gu, Bingke Zhu, Guibo Zhu, Yingying Chen et al.AAAI 2024 · 312 citations
- MMR-AD: A Large-Scale Multimodal Dataset for Benchmarking General Anomaly Detection with Multimodal Large Language ModelsXincheng Yao, Zefeng Qian, Chao Shi, Jiayang Song et al.CVPR 2026 · 2 citations
- Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language ModelsJiacong Xu, Shao-Yuan Lo, Bardia Safaei, Vishal M. Patel et al.CVPR 2025
- Semantic Curriculum for Anomaly Detection: A Unified Language-Driven Meta-Optimization FrameworkKai Tan, Yangliu Du, Dongyang Zhan, Haining Yu et al.INFOCOM 2026
