AutoUAD: Hyper-parameter Optimization for Unsupervised Anomaly Detection

Unsupervised anomaly detection (UAD) has important applications in diverse fields such as manufacturing industry and medical diagnosis. In the past decades, although numerous insightful and effective UAD methods have been proposed, it remains a huge challenge to tune the hyper-parameters of each method and select the most appropriate method among many candidates for a specific dataset, due to the absence of labeled anomalies in the training phase of UAD methods and the high diversity of real datasets. In this work, we aim to address this challenge, so as to make UAD more practical and reliable. We propose two internal evaluation metrics, relative-top-median and expected-anomaly-gap, and one semi-internal evaluation metric, normalized pseudo discrepancy (NPD), as surrogate functions of the expected model performance on unseen test data. For instance, NPD measures the discrepancy between the anomaly scores of a validation set drawn from the training data and a validation set drawn from an isotropic Gaussian. NPD is simple and hyper-parameter-free and is able to compare different UAD methods, and its effectiveness is theoretically analyzed. We integrate the three metrics with Bayesian optimization to effectively optimize the hyper-parameters of UAD models. Extensive experiments on 38 datasets show the effectiveness of our methods.

• Corresponding author 1 UAD assumes all or most of the training data are from normal conditions of a system. See Definition 1.

To address these challenges, we first propose two new internal evaluation metrics, relative-topkmedian and expected-anomaly-gap, under two proper assumptions in UAD. In the empirical studies, we found that these two metrics do not always work well and are not effective in comparing different UAD methods because they could overfit the training data when implemented on a complex UAD model, and they both have an additional hyper-parameter to determine in advance, as the assumption defined. We then propose a semi-internal evaluation metric, normalized pseudo discrepancy (NPD), which measures the discrepancy between the anomaly scores of a validation set drawn from the training data and a validation set drawn from an isotropic Gaussian distribution. It offers more robust and reliable results under simpler assumptions without additional hyper-parameters. Our metrics provide a more nuanced understanding of model performance based on a reliable theoretical guarantee, enabling a better selection of hyper-parameters and models in UAD tasks.

Aiming at model and hyper-parameter selection for UAD algorithms and improving the convenience, accuracy, and efficiency of UAD, our contributions are highlighted as follows.

• We propose two internal evaluation metrics, relative-top-median and expected-anomaly-gap, and one semi-internal evaluation metric, normalized pseudo discrepancy, for automated UAD.

• We implement automated UAD using Bayesian optimization. It automatically and efficiently selects the possibly best hyper-parameters guided by our proposed metrics.

• We provide theoretical guarantees for our NPD metric to ensure feasibility and reliability.