Start Smart: Leveraging Gradients For Enhancing Mask-based XAI Methods

Mask-based explanation methods offer a powerful framework for interpreting deep learning model predictions across diverse data modalities, such as images and time series, in which the central idea is to identify an instance-dependent mask that minimizes the performance drop from the resulting masked input. Different objectives for learning such masks have been proposed, all of which, in our view, can be unified under an information-theoretic framework that balances performance degradation of the masked input with the complexity of the resulting masked representation. Typically, these methods initialize the masks either uniformly or as all-ones. In this paper, we argue that an effective mask initialization strategy is as important as the development of novel learning objectives, particularly in light of the significant computational costs associated with existing mask-based explanation methods. To this end, we introduce a new gradient-based initialization technique called StartGrad, which is the first initialization method specifically designed for mask-based post-hoc explainability methods. Compared to commonly used strategies, StartGrad is provably superior at initialization in striking the aforementioned trade-off. Despite its simplicity, our experiments demonstrate that StartGrad enhances the optimization process of various state-ofthe-art mask-explanation methods by reaching target metrics faster and, in some cases, boosting their overall performance.

Published as a conference paper at ICLR 2025 especially when compared to gradient-based saliency techniques like SmoothGrad (Smilkov et al., 2017) or Integrated Gradients (Sundararajan et al., 2017). For instance, the most advanced maskbased methods in the vision domain such as the recently proposed WaveletX (Kolek et al., 2023) and ShearletX (Kolek et al., 2023), require orders of magnitude more execution time to generate explanations than their gradient-based counterparts. This trade-off between performance and speed presents a major limitation for mask-based methods, especially in time-sensitive applications where rapid decisions are critical. As a result, users are often forced to choose between the superior faithfulness of mask-based explanations and the faster, but potentially less reliable, gradient-based alternatives. This tension between explanation accuracy and real-time applicability remains a key challenge in making mask-based methods more widely applicable in high-stakes environments such as healthcare.

To address the challenge of balancing performance and computational efficiency in mask-based methods, we draw on two key insights: First, while the way how to initialize masks in existing mask-based XAI methods is usually neglected, it plays a crucial role in optimization in terms of both running time and the final achievable maximum or minimum value. Second, although gradient-based saliency methods are not explicitly designed to meet desiderata of high-quality explanations, they do provide valuable signals about the model's decision-making process with minimal computational overhead. By combining these two insights, we propose StartGrad, a novel gradient-based mask initialization technique specifically designed for post-hoc explanation methods. StartGrad leverages gradient signals to provide provably superior initialization masks in terms of minimal distortion and sparsity-two essential criteria for mask-based explanation methods-compared to commonly used strategies. By doing so, StartGrad harnesses the strengths of gradient-based approaches to enhance existing mask-based explainability techniques.

We summarize our contributions as below:

• We introduce StartGrad, a novel gradient-based mask initialization algorithm grounded in the rate distortion explanation (RDE) framework (Macdonald et al., 2019), representing the first initialization technique explicitly designed to enhance the performance of mask-based explanation methods.

• We prove that StartGrad is superior at initialization compared to other initialization strategies in reducing distortion and improving sparsity-two essential criteria for effective mask-based explanations.

• Extensive experiments on vision and time-series tasks demonstrate that StartGrad enables state-of-the-art methods like ShearletX (Kolek et al., 2023) and ExtremalMask (Enguehard, 2023) to reach target metrics faster while also improving overall performance in some cases.

• To the best of our knowledge, this work presents the first comprehensive theoretical and empirical analysis of mask initialization techniques across both vision and time-series domains, providing critical insights for improving mask-based explanation methods.

Feature attribution methods can be divided into white-box, gray-box, and black-box approaches, depending on the amount of information required to generate an explanation (Muzellec et al., 2024). Mask-based explanation methods are considered black-box attribution techniques, as they