Lune

ICCV2025Top-tier venue

SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutions

Jessica Bader, Leander Girrbach, Stephan Alaniz, Zeynep Akata

Abstract

Concept Bottleneck Models (CBMs) and other concept-based interpretable models show great promise for making AI applications more transparent, which is essential in fields like medicine. Despite their success, we demonstrate that CBMs struggle to reliably identify the correct concepts under distribution shifts. To assess the robustness of CBMs to concept variations, we introduce SUB: a fine-grained image and concept benchmark containing 38,400 synthetic images based on the CUB dataset. To create SUB, we select a CUB subset of 33 bird classes and 45 concepts to generate images which substitute a specific concept, such as wing color or belly pattern. We introduce a novel Tied Diffusion Guidance (TDG) method to precisely control generated images, where noise sharing for two parallel denoising processes ensures that both the correct bird class and the correct attribute are generated. This novel benchmark enables rigorous evaluation of CBMs and similar interpretable models, contributing to the development of more robust methods. Our code is available at https://github.com/ExplainableML/sub and the dataset at http://huggingface.co/datasets/Jessica-bader/SUB.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models cites Sigmoid Loss for Language Image Pre-Training
  • Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models cites Learning Transferable Visual Models From Natural Language Supervision
  • SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis cites High-Resolution Image Synthesis with Latent Diffusion Models
  • SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis cites Adding Conditional Control to Text-to-Image Diffusion Models
  • SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis cites Learning Transferable Visual Models From Natural Language Supervision
  • Adding Conditional Control to Text-to-Image Diffusion Models cites High-Resolution Image Synthesis with Latent Diffusion Models
  • Adding Conditional Control to Text-to-Image Diffusion Models cites Learning Transferable Visual Models From Natural Language Supervision
  • Sigmoid Loss for Language Image Pre-Training cites Learning Transferable Visual Models From Natural Language Supervision
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Sparse Autoencoders Learn Monosemantic Features in Vision-Language ModelsMateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge J. Belongie et al.NeurIPS 2025 · 79 citations

Builds on34

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
  3. Adding Conditional Control to Text-to-Image Diffusion ModelsLvmin Zhang, Anyi Rao, Maneesh AgrawalaICCV 2023 · 6,759 citations
  4. SDXL: Improving Latent Diffusion Models for High-Resolution Image SynthesisDustin Podell, Zion English, Kyle Lacey, Andreas Blattmann et al.ICLR 2024 · 4,569 citations
  5. Sigmoid Loss for Language Image Pre-TrainingXiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, Lucas BeyerICCV 2023 · 2,932 citations

Related papers

  1. Concept-Based Unsupervised Domain AdaptationXinyue Xu, Yueying Hu, Hui Tang, Yi Qin et al.ICML 2025
  2. Auxiliary Losses for Learning Generalizable Concept-based ModelsIvaxi Sheth, Samira Ebrahimi KahouNeurIPS 2023 · 52 citations
  3. Interpretable Generative Models through Post-hoc Concept BottlenecksAkshay R. Kulkarni, Ge Yan, Chung-En Sun, Tuomas P. Oikarinen et al.CVPR 2025
  4. Beyond Concept Bottleneck Models: How to Make Black Boxes Intervenable?Sonia Laguna, Ricards Marcinkevics, Moritz Vandenhirtz, Julia E. VogtNeurIPS 2024 · 39 citations
  5. Vision-Language Models Encode Clinical Guidelines for Concept-Based Medical ReasoningMohamed Harmanani, Bining Long, Zhuoxin Guo, Paul F.R. Wilson, Amirhossein Sabour, Minh Nguyen Nhat To, Gabor Fichtinger, Purang Abolmaesumi, Parvin MousaviCVPR Findings 2026
Dusk over the sea between two cliffs drawn in fine vertical lines