LHAct: Rectifying Extremely Low and High Activations for Out-of-Distribution Detection

In recent years, out-of-distribution (OOD) detection has emerged as a crucial research area, especially when deploying AI products in real-world scenarios. OOD detection researchers have made significant efforts to mitigate the adverse effects of abnormal activation values (abbr. activations) that refer to the outputs of the activation function acted on feature maps. Since abnormal activations would cause difficulty in separating ID and OOD data, the previous unified solution is to rectify the extremely high abnormal activations by clipping them with a pre-defined threshold or filtering them with a low-pass filter. However, it ignores the extremely low abnormal activations, and the proposed rectification strategy is always suboptimal because the used rectification function is non-convergence or high-intensity convergence, leading to under-rectification or over-rectification. In this paper, we propose an approach called Rectifying Extremely Low and High Activations (LHAct). LHAct includes a newly-designed function to rectify the extremely low and high activations at the same time. Specifically, LHAct increases the difference of means between ID and OOD activation distributions while decreasing their variances after processing the original activations. Our theoretical analyses demonstrate that LHAct significantly enhances the separability of ID and OOD data. By conducting extensive experiments, we demonstrate that LHAct surpasses previous activation-based methods significantly and generalizes well to other architectures and OOD scores. Code is available at: https://github.com/ystyuan/LHAct.git.