Lune

ICML2025Top-tier venue

From Pixels to Perception: Interpretable Predictions via Instance-wise Grouped Feature Selection

Moritz Vandenhirtz, Julia E. Vogt

Abstract

Understanding the decision-making process of machine learning models provides valuable insights into the task, the data, and the reasons behind a model's failures. In this work, we propose a method that performs inherently interpretable predictions through the instance-wise sparsification of input images. To align the sparsification with human perception, we learn the masking in the space of semantically meaningful pixel regions rather than on pixel-level. Additionally, we introduce an explicit way to dynamically determine the required level of sparsity for each instance. We show empirically on semi-synthetic and natural image datasets that our inherently interpretable classifier produces more meaningful, human-understandable predictions than state-ofthe-art benchmarks.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on17

  1. Searching for MobileNetV3Andrew Howard, Ruoming Pang, Hartwig Adam, Quoc V. Le et al.ICCV 2019 · 9,163 citations
  2. Training data-efficient image transformers & distillation through attentionHugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa et al.ICML 2021 · 8,974 citations
  3. Scaling Rectified Flow Transformers for High-Resolution Image SynthesisPatrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari et al.ICML 2024 · 3,620 citations
  4. Concept Bottleneck ModelsPang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann et al.ICML 2020 · 1,233 citations
  5. Noise or Signal: The Role of Image Backgrounds in Object RecognitionKai Yuanqing Xiao, Logan Engstrom, Andrew Ilyas, Aleksander MadryICLR 2021 · 451 citations

Related papers

  1. Model Interpretability and Rationale Extraction by Input Mask OptimizationMarc Felix Brinner, Sina ZarrießACL Findings 2023 · 2 citations
  2. Learning Decision Trees Recurrently Through CommunicationStephan Alaniz, Diego Marcos, Bernt Schiele, Zeynep AkataCVPR 2021
  3. Passive attention in artificial neural networks predicts human visual selectivityThomas A. Langlois, H. Charles Zhao, Erin Grant, Ishita Dasgupta et al.NeurIPS 2021 · 19 citations
  4. Logic Rule Guided Attribution with Dynamic AblationJianqiao An, Yuandu Lai, Yahong HanAAAI 2022 · 4 citations
  5. What You See is What You Classify: Black Box AttributionsSteven Stalder, Nathanaël Perraudin, Radhakrishna Achanta, Fernando Pérez-Cruz et al.NeurIPS 2022 · 15 citations
Dusk over the sea between two cliffs drawn in fine vertical lines