AutoDebias: An Automated Framework for Detecting and Mitigating Backdoor Biases in Text-to-Image Models
Hongyi Cai, Mohammad Mahdinur Rahman, MingKang Dong, Muxin Pu, Moayad Aloqaily, jie li, Xinfeng Li, Jialie Shen, Meikang Qiu, Qingsong Wen
Abstract
Text-to-Image (T2I) models generate high-quality images but are vulnerable to malicious backdoor attacks that inject harmful biases (e.g., trigger-activated gender or racial stereotypes). Existing debiasing methods, often designed for natural statistical biases, struggle with these deliberate and subtle injected attacks. We propose AutoDebias, a framework that automatically identifies and mitigates these malicious biases in T2I models without prior knowledge of the specific attack vectors. Specifically, AutoDebias leverages vision-language models to detect trigger-activated visual patterns and constructs neutralization guides by generating counter-prompts. These guides drive a CLIP-guided training process that breaks the harmful associations while preserving the original model's image quality and diversity. Unlike methods designed for natural bias, AutoDebias effectively addresses subtle, injected stereotypes and multiple interacting attacks. We evaluate the framework on a new benchmark covering 17 distinct backdoor attack scenarios, including challenging cases where multiple backdoors co-exist. AutoDebias detects malicious patterns with 91.6% accuracy and reduces the backdoor success rate from 90% to negligible levels, while preserving the visual fidelity of the original model.
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Related papers
- MTAttack: Multi-Target Backdoor Attacks Against Large Vision-Language ModelsZihan Wang, Guansong Pang, Wenjun Miao, Jin Zheng et al.AAAI 2026
- Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data PoisoningShengfang Zhai, Yinpeng Dong, Qingni Shen, Shi Pu et al.ACM MM 2023 · 46 citations
- Exposing Hidden Biases in Text-to-Image Models via Automated Prompt SearchManos Plitsis, Giorgos Bouritsas, Vassilis Katsouros, Yannis PanagakisICML 2026
- Association of Objects May Engender Stereotypes: Mitigating Association-Engendered Stereotypes in Text-to-Image GenerationJunlei Zhou, Jiashi Gao, Xiangyu Zhao, Xin Yao et al.NeurIPS 2024 · 5 citations
- Towards Human-Imperceptible Backdoor Attacks on Text-to-Image Diffusion ModelsChangkun Wu, Chenghao Chen, Wu kun, Chong Fu et al.CVPR 2026
