CRAFT: Concept Recursive Activation FacTorization for Explainability

Figure 1 . The "Man on the Moon" incorrectly classified as a "shovel" by an ImageNet-trained ResNet50. Heatmap generated by a classic attribution method [55] (left) vs. concept attribution maps generated with the proposed CRAFT approach (right) which highlights the two most influential concepts that drove the ResNet50's decision along with their corresponding locations. CRAFT suggests that the neural net arrived at its decision because it identified the concept of "dirt" • commonly found in members of the image class "shovel" and the concept of "ski pants" • typically worn by people clearing snow from their driveway with a shovel instead the correct concept of astronaut's pants (which was probably never seen during training).