Efficient Video Anomaly Detection via Scene-Dependent Memory Assisted Inter-Frame RGB Difference Reconstruction

In video anomaly detection task, to mitigate the interference of background noise on learning the appearance and motion features of foreground objects, existing object-centric methods often directly disregard scene information, making it challenging for them to detect scene-dependent anomalies. Moreover, we observe that most methods focus on reconstructing or predicting complete frame-level or object-level RGB information, which limits their inference speed. In this work, we propose a novel inter-frame RGB difference reconstruction network for efficient video anomaly detection. Specifically, we construct separate scene-dependent memory banks (SDMBs) for different scenes to store exclusive normal patterns, thus enabling sensitive detection of scene-dependent anomalies. Meanwhile, we design sparse aggregation and similarity-driven updating mechanisms for the memory items in the SDMBs, which effectively increase the reconstruction error of anomalies by adequately learning the diverse normal patterns in both the training and testing data, thus making the distinction between normal and abnormal frames easier. Extensive experiments on three public datasets demonstrate that our method outperforms state-of-the-art approaches in terms of detection accuracy, false negative rate, and inference speed, particularly in situations with more complex scene and event types.