Navigating Neural Space: Revisiting Concept Activation Vectors to Overcome Directional Divergence

With a growing interest in understanding neural network prediction strategies, Concept Activation Vectors (CAVs) have emerged as a popular tool for modeling human-understandable concepts in the latent space. Commonly, CAVs are computed by leveraging linear classifiers optimizing the separability of latent representations of samples with and without a given concept. However, in this paper we show that such a separability-oriented computation leads to solutions, which may diverge from the actual goal of precisely modeling the concept direction. This discrepancy can be attributed to the significant influence of distractor directions, i.e., signals unrelated to the concept, which are picked up by filters (i.e., weights) of linear models to optimize class-separability. To address this, we introduce pattern-based CAVs, solely focussing on concept signals, thereby providing more accurate concept directions. We evaluate various CAV methods in terms of their alignment with the true concept direction and their impact on CAV applications, including concept sensitivity testing and model correction for shortcut behavior caused by data artifacts. We demonstrate the benefits of pattern-based CAVs using the Pediatric Bone Age, ISIC2019, and FunnyBirds datasets with VGG, ResNet, ReXNet, EfficientNet, and Vision Transformer as model architectures. 1 . RELATED WORK A variety of approaches has emerged to identify human-understandable concepts in DNNs. Some works consider single neurons as concepts (Olah et al., 2017; Achtibat et al., 2023) , while others focus on identifying interesting subspaces (Vielhaben et al., 2023) or linear directions (Nanda et al., 2023) . We follow the latter approach and encode concepts as linear combinations of neurons, also known as superposition (Elhage et al., 2022) . These directions can be identified through unsupervised activation matrix factorization (Fel et al., 2023) or by the supervised training of CAVs, i.e., vectors pointing from samples without to samples with the concept. In the absence of concept labels, automated concept discovery approaches can further streamline this process (Ghorbani et al., 2019; Zhang et al., 2021) . Various methods leverage CAVs as latent concept representation. For instance, TCAV measures a