Diffusion-CAM: Faithful Visual Explanations for dMLLMs
Haomin Zuo, Yidi Li, Luoxiao Yang, Xiaofeng Zhang
摘要
While diffusion Multimodal Large Language Models (dMLLMs) have recently achieved remarkable strides in multimodal generation, the development of interpretability mechanisms has lagged behind their architectural evolution. Unlike traditional autoregressive models that produce sequential activations, diffusion-based architectures generate tokens via parallel denoising, resulting in smooth, distributed activation patterns across the entire sequence. Consequently, existing Class Activation Mapping (CAM) methods, which are tailored for local, sequential dependencies, are ill-suited for interpreting these non-autoregressive behaviors. To bridge this gap, we propose Diffusion-CAM, the first interpretability method specifically tailored for dMLLMs. We derive raw activation maps by differentiably probing intermediate representations in the transformer backbone, accordingly capturing both latent features and their class-specific gradients. To address the inherent stochasticity of these raw signals, we incorporate four key modules to resolve spatial ambiguity and mitigate intra-image confounders and redundant token correlations. Extensive experiments demonstrate that Diffusion-CAM significantly outperforms SoTA methods in both localization accuracy and visual fidelity, establishing a new standard for understanding the parallel generation process of diffusion multimodal systems. Code is available at https:// github.com/ZzzzzZhhmm/Diffusion-CAM
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper30
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- Denoising Diffusion Implicit ModelsJiaming Song, Chenlin Meng, Stefano ErmonICLR 2021 · 被引用 11,743 次
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 被引用 11,349 次
- Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech 等NeurIPS 2022 · 被引用 6,707 次
- Large Language Diffusion ModelsShen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang 等NeurIPS 2025 · 被引用 949 次
相关 Paper
- Token Activation Map to Visually Explain Multimodal LLMsYi Li, Hualiang Wang, Xinpeng Ding, Haonan Wang 等ICCV 2025 · 被引用 3 次
- D3ToM: Decider-Guided Dynamic Token Merging for Accelerating Diffusion MLLMsShuochen Chang, Xiaofeng Zhang, Qingyang Liu, Li NiuAAAI 2026
- Generative Multimodal Pretraining with Discrete Diffusion Timestep TokensKaihang Pan, Wang Lin, Zhongqi Yue, Tenglong Ao 等CVPR 2025
- DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse AutoencodersXu Wang, Bingqing Jiang, Yu Wan, Baosong Yang 等ICML 2026
- DiffCAM: Data-Driven Saliency Maps by Capturing Feature DifferencesXingjian Li, Qiming Zhao, Neelesh Bisht, Mostofa Rafid Uddin 等CVPR 2025
