Quantifying Statistical Significance of Deep Nearest Neighbor Anomaly Detection via Selective Inference
Mizuki Niihori, Shuichi Nishino, Teruyuki Katsuoka, Tomohiro Shiraishi, Kouichi Taji, Ichiro Takeuchi
Abstract
In real-world applications, anomaly detection (AD) often operates without access to anomalous data, necessitating semi-supervised methods that rely solely on normal data. Among these methods, deep k-nearest neighbor (deep kNN) AD stands out for its interpretability and flexibility, leveraging distance-based scoring in deep latent spaces. Despite its strong performance, deep kNN lacks a mechanism to quantify uncertainty-an essential feature for critical applications such as industrial inspection. To address this limitation, we propose a statistical framework that quantifies the significance of detected anomalies in the form of p-values, thereby enabling control over false positive rates at a user-specified significance level (e.g.,0.05). A central challenge lies in managing selection bias, which we tackle using Selective Inference-a principled method for conducting inference conditioned on data-driven selections. We evaluate our method on diverse datasets and demonstrate that it provides reliable AD well-suited for industrial use cases.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext d40aae30-fe54-4bab-88b4-73d46d252584Builds on6
- Computing Valid p-value for Optimal Changepoint by Selective Inference using Dynamic ProgrammingVo Nguyen Le Duy, Hiroki Toda, Ryota Sugiyama, Ichiro TakeuchiNeurIPS 2020 · 45 citations
- Fast and More Powerful Selective Inference for Sparse High-Order Interaction ModelDiptesh Das, Vo Nguyen Le Duy, Hiroyuki Hanada, Koji Tsuda et al.AAAI 2022 · 23 citations
- Quantifying Statistical Significance of Neural Network-based Image Segmentation by Selective InferenceVo Nguyen Le Duy, Shogo Iwazaki, Ichiro TakeuchiNeurIPS 2022 · 21 citations
- Statistical Test for Attention Maps in Vision TransformersTomohiro Shiraishi, Daiki Miwa, Teruyuki Katsuoka, Vo Nguyen Le Duy et al.ICML 2024 · 7 citations
- Valid P-Value for Deep Learning-driven Salient RegionDaiki Miwa, Vo Nguyen Le Duy, Ichiro TakeuchiICLR 2023 · 3 citations
Related papers
- Deep Semi-Supervised Anomaly DetectionLukas Ruff, Robert A. Vandermeulen, Nico Görnitz, Alexander Binder et al.ICLR 2020 · 678 citations
- Computing Valid P-Values for Image Segmentation by Selective InferenceKosuke Tanizaki, Noriaki Hashimoto, Yu Inatsu, Hidekata Hontani et al.CVPR 2020
- Statistical Test for Feature Selection Pipelines by Selective InferenceTomohiro Shiraishi, Tatsuya Matsukawa, Shuichi Nishino, Ichiro TakeuchiICML 2025
- ELITE: Robust Deep Anomaly Detection with Meta GradientHuayi Zhang, Lei Cao, Peter M. VanNostrand, Samuel Madden et al.KDD 2021 · 17 citations
- ANEDL: Adaptive Negative Evidential Deep Learning for Open-Set Semi-supervised LearningYang Yu, Danruo Deng, Furui Liu, Qi Dou et al.AAAI 2024 · 5 citations
