Generating Anomalies for Video Anomaly Detection with Prompt-based Feature Mapping
Zuhao Liu, Xiao-Ming Wu, Dian Zheng, Kun-Yu Lin, Wei-Shi Zheng
Abstract
Anomaly detection in surveillance videos is a challenging computer vision task where only normal videos are available during training. Recent work released the first virtual anomaly detection dataset to assist real-world detection. However, an anomaly gap exists because the anomalies are bounded in the virtual dataset but unbounded in the real world, so it reduces the generalization ability of the virtual dataset. There also exists a scene gap between virtual and real scenarios, including scene-specific anomalies (events that are abnormal in one scene but normal in another) and scene-specific attributes, such as the viewpoint of the surveillance camera. In this paper, we aim to solve the problem of the anomaly gap and scene gap by proposing a prompt-based feature mapping framework (PFMF). The PFMF contains a mapping network guided by an anomaly prompt to generate unseen anomalies with unbounded types in the real scenario, and a mapping adaptation branch to narrow the scene gap by applying domain classifier and anomaly classifier. The proposed framework outperforms the state-of-the-art on three benchmark datasets. Extensive ablation experiments also show the effectiveness of our framework design.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
11 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction cites Anomaly Detection in Video Sequence With Appearance-Motion Correspondence
- Multi-Scale Video Anomaly Detection by Multi-Grained Spatio-Temporal Representation Learning cites A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction
- Multi-Scale Video Anomaly Detection by Multi-Grained Spatio-Temporal Representation Learning cites Anomaly Detection in Video Sequence With Appearance-Motion Correspondence
- Towards Multi-Domain Learning for Generalizable Video Anomaly Detection cites Anomaly Detection in Video Sequence With Appearance-Motion Correspondence
- Conditional Prompt Learning for Vision-Language Models cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors cites A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction
- Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors cites Anomaly Detection in Video Sequence With Appearance-Motion Correspondence
- Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Open-Vocabulary Video Anomaly Detection cites A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction
- Open-Vocabulary Video Anomaly Detection cites Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors
- Open-Vocabulary Video Anomaly Detection cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Cited by top-tier papers11
- Open-Vocabulary Video Anomaly DetectionPeng Wu, Xuerong Zhou, Guansong Pang, Yujia Sun et al.CVPR 2024 · 56 citations
- Generative Semi-supervised Graph Anomaly DetectionHezhe Qiao, Qingsong Wen, Xiaoli Li, Ee-Peng Lim et al.NeurIPS 2024 · 48 citations
- Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly DetectorsNicolae-Catalin Ristea, Florinel-Alin Croitoru, Radu Tudor Ionescu, Marius Popescu et al.CVPR 2024 · 47 citations
- Multi-Scale Video Anomaly Detection by Multi-Grained Spatio-Temporal Representation LearningMenghao Zhang, Jingyu Wang, Qi Qi, Haifeng Sun et al.CVPR 2024 · 29 citations
- Towards Multi-Domain Learning for Generalizable Video Anomaly DetectionMyeongAh Cho, Taeoh Kim, Minho Shim, Dongyoon Wee et al.NeurIPS 2024 · 14 citations
Builds on18
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- Conditional Prompt Learning for Vision-Language ModelsKaiyang Zhou, Jingkang Yang, Chen Change Loy, Ziwei LiuCVPR 2022 · 1,438 citations
- Anomaly Detection in Video Sequence With Appearance-Motion CorrespondenceTrong-Nguyen Nguyen, Jean MeunierICCV 2019 · 414 citations
- A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame PredictionZhian Liu, Yongwei Nie, Chengjiang Long, Qing Zhang et al.ICCV 2021 · 341 citations
- Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language ModelYu Du, Fangyun Wei, Zihe Zhang, Miaojing Shi et al.CVPR 2022 · 311 citations
Related papers
- Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly DetectionZhiwei Yang, Jing Liu, Peng WuCVPR 2024 · 55 citations
- Harnessing Large Language Models for Training-Free Video Anomaly DetectionLuca Zanella, Willi Menapace, Massimiliano Mancini, Yiming Wang et al.CVPR 2024 · 57 citations
- A New Comprehensive Benchmark for Semi-supervised Video Anomaly Detection and AnticipationCongqi Cao, Yue Lu, Peng Wang, Yanning ZhangCVPR 2023
- Prompt-Enhanced Multiple Instance Learning for Weakly Supervised Video Anomaly DetectionJunxi Chen, Liang Li, Li Su, Zheng-Jun Zha et al.CVPR 2024
- MGFN: Magnitude-Contrastive Glance-and-Focus Network for Weakly-Supervised Video Anomaly DetectionYingxian Chen, Zhengzhe Liu, Baoheng Zhang, Wilton W. T. Fok et al.AAAI 2023 · 221 citations
