Towards Trustworthy Video Anomaly Understanding: A Class-Guided Chain-of-Evaluation Metric and An Anomaly-focused Meta-Benchmark
Jiaxu Leng, Zhoujie Huang, Mingpi Tan, Zhanjie Wu, Xinbo Gao
Abstract
The trustworthiness of evaluation is critical to reliable model comparison and deployment in Video Anomaly Understanding (VAU). However, existing metrics are sensitive to expression styles and normal content, and this field lacks a diagnostic benchmark to validate metric validity and robustness. To bridge this gap, we propose: (1) a Class-Guided Chain-of-Evaluation (CG-CoE) metric, which structures assessment by extracting anomalous events and matching them under a class-specific semantic tolerance boundary, thereby decoupling anomaly semantics from expression styles; and (2) an anomaly-focused metaevaluation benchmark with two subsets: Anomalous Event-level Annotations (AEA) subset for measuring the validity of reflecting VAU models' anomaly understanding ability and Controlled Variant Pairs (CVP) with fixed anomalies for quantifying robustness to stylistic perturbations. Extensive experiments demonstrate that CG-CoE achieves state-of-the-art (SOTA) validity and robustness.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on15
- Towards a Unified Multi-Dimensional Evaluator for Text GenerationMing Zhong, Yang Liu, Da Yin, Yuning Mao et al.EMNLP 2022 · 103 citations
- Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-ThoughtChao Huang, Benfeng Wang, Wei Wang, Jie Wen et al.NeurIPS 2025 · 30 citations
- Beyond Euclidean: Dual-Space Representation Learning for Weakly Supervised Video Violence DetectionJiaxu Leng, Zhanjie Wu, Mingpi Tan, Yiran Liu et al.NeurIPS 2024 · 22 citations
- Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie CalibrationDaniel Deutsch, George F. Foster, Markus FreitagEMNLP 2023 · 14 citations
- Anomaly Warning: Learning and Memorizing Future Semantic Patterns for Unsupervised Ex-ante Potential Anomaly PredictionJiaxu Leng, Mingpi Tan, Xinbo Gao, Wen Lu et al.ACM MM 2022 · 7 citations
Related papers
- VALU: A Benchmark for Video Anomaly Temporal Localization and Understanding at Multiple Semantic LevelsYixiao He, Menghao Zhang, Haifeng Sun, Jing Wang et al.ACL 2026
- Uncovering what, why and How: A Comprehensive Benchmark for Causation Understanding of Video AnomalyHang Du, Sicheng Zhang, Binzhu Xie, Guoshun Nan et al.CVPR 2024
- CueBench: Advancing Unified Understanding of Context-Aware Video Anomalies in Real-WorldYating Yu, Congqi Cao, Zhaoying Wang, Weihua Meng et al.AAAI 2026 · 1 citation
- FineVAU: A Novel Human-Aligned Benchmark for Fine-Grained Video Anomaly UnderstandingJoão Alexandre Cardeira Pereira, Vasco Lopes, João Neves, David SemedoAAAI 2026
- VAGU & GtS: LLM-Based Benchmark and Framework for Joint Video Anomaly Grounding and UnderstandingShibo Gao, Peipei Yang, Yangyang Liu, Yi Chen et al.AAAI 2026 · 5 citations
