Towards Global Explanations of Convolutional Neural Networks With Concept Attribution

With the growing prevalence of convolutional neural networks (CNNs), there is an urgent demand to explain their behaviors. Global explanations contribute to understanding model predictions on a whole category of samples, and thus have attracted increasing interest recently. However, existing methods overwhelmingly conduct separate input attribution or rely on local approximations of models, making them fail to offer faithful global explanations of CNNs. To overcome such drawbacks, we propose a novel two-stage framework, Attacking for Interpretability (AfI), which explains model decisions in terms of the importance of userdefined concepts. AfI first conducts a feature occlusion analysis, which resembles a process of attacking models to derive the category-wide importance of different features. We then map the feature importance to concept importance through ad-hoc semantic tasks. Experimental results confirm the effectiveness of AfI and its superiority in providing more accurate estimations of concept importance than existing proposals. * Corresponding author. 1 To avoid confusion, we consistently use the term "feature" to refer to the visual patterns detected by feature filters of CNNs (e.g., the banded texture), rather than the input pixels. 8649 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)