Closed-Form Concept Erasure via Double Projections
Chi Zhang, Jingpu Cheng, Zhixian Wang, Ping Liu
Abstract
While modern generative models such as diffusion-based architectures have enabled impressive creative capabilities, they also raise important safety and ethical risks. These concerns have led to growing interest in concept erasure, the process of removing unwanted concepts from model representations. Existing approaches often achieve strong erasure performance but rely on iterative optimization and may inadvertently distort unrelated concepts. In this work, we present a simple yet principled alternative: a linear transformation framework that achieves concept erasure analytically, without any training. Our method adapts a pretrained model through two sequential, closed-form steps: first, computing a proxy projection of the target concept, and second, applying a constrained transformation within the left null space of known concept directions. This design yields a deterministic and geometrically interpretable procedure for safe, efficient, and theory-grounded concept removal. Across a wide range of experiments, including object and style erasure on multiple Stable Diffusion variants and the flow-matching model (FLUX), our approach matches or surpasses the performance of state-of-the-art methods while preserving non-target concepts more faithfully. Requiring only a few seconds to apply, it offers a lightweight and drop-in tool for controlled model editing, advancing the goal of safer and more responsible generative models. Code is available here.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 7b11a336-0a9f-4e82-b230-fa9a0ae5386bCited by top-tier papers1
Ask how each one uses itBuilds on38
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
Related papers
- GEM: Geometric Erasure by Contrastive Velocity Matching in Rectified FlowsJonas Henry Grebe, Tobias Braun, Anna Rohrbach, Marcus RohrbachICML 2026 · 1 citation
- EraseAnything: Enabling Concept Erasure in Rectified Flow TransformersDaiheng Gao, Shilin Lu, Wenbo Zhou, Jiaming Chu et al.ICML 2025
- Orthogonal Concept Erasure for Diffusion ModelsYuhao Sun, Lingyun Yu, Hao-Xiang Xu, Fengyuan Miao et al.ICML 2026
- Minimalist Concept Erasure in Generative ModelsYang Zhang, Er Jin, Yanfei Dong, Yixuan Wu et al.ICML 2025
- ACE: Concept Editing in Diffusion Models without Performance DegradationRuipeng Wang, Junfeng Fang, Jiaqi Li, Hao Chen et al.ACM MM 2025
