Filter-Guided Diffusion for Controllable Image Generation
Zeqi Gu, Ethan Yang, Abe Davis
Abstract
3 σ value σ spatial σ value Generation with Guidance from Corresponding Bilateral Parameters Decoded Guide Structure for Different Bilateral Parameters Sampling with Different FGD Parameters Yields Higher-Quality Results in Less Time Than it Takes Previous SOTA Methods to Generate One Output Guidance Strength Figure 1: We present Filter-Guided Diffusion (FGD), a training-free black box method for conditioning image diffusion on the structure of example images. FGD works by adding a fast filtering step between each iteration of the diffusion process. Building on classic image processing theory, we can design this filtering step to preserve the structure of a given guide image. The grid on the far left shows the effects of classic bilateral filters with different parameters on an image. The middle grid shows the result of using the same filters for guidance with FGD (in this case, using an empty prompt). The spatial filter size, which controls the scale of blur in a traditional bilateral, determines the spatial scale at which the network may vary from structure of the input image. The value filter size, which controls edge preservation in the traditional bilateral, determines how much the diffusion process should respect the edges of the guide. In addition to the filter's parameters, we can control the overall strength of guidance to determine how closely the result should resemble the input. The right shows two examples of image-to-image translation with varying guidance strength. Results become closer to the guide image as guidance strength increases to the right.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- High-Resolution Image Synthesis with Latent Diffusion Models cites Denoising Diffusion Probabilistic Models
- Generative Photomontage cites High-Resolution Image Synthesis with Latent Diffusion Models
- Generative Photomontage cites Denoising Diffusion Probabilistic Models
- Directly Denoising Diffusion Models cites High-Resolution Image Synthesis with Latent Diffusion Models
- Directly Denoising Diffusion Models cites Denoising Diffusion Probabilistic Models
- Learning Transferable Visual Models From Natural Language Supervision cites Language Models are Few-Shot Learners
Cited by top-tier papers2
Builds on17
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Directly Denoising Diffusion ModelsDan Zhang, Jingjing Wang, Feng LuoICML 2024 · 11,724 citations
Related papers
- Frequency-Guided Diffusion for Training-Free Text-Driven Image TranslationZheng Gao, Jifei Song, Zhensong Zhang, Jiankang Deng et al.ICCV 2025 · 1 citation
- Guiding a Diffusion Model with a Bad Version of ItselfTero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen et al.NeurIPS 2024 · 338 citations
- On Distillation of Guided Diffusion ModelsChenlin Meng, Robin Rombach, Ruiqi Gao, Diederik P. Kingma et al.CVPR 2023
- Manifold Preserving Guided DiffusionYutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida et al.ICLR 2024 · 148 citations
- Frequency-Controlled Diffusion Model for Versatile Text-Guided Image-to-Image TranslationXiang Gao, Zhengbo Xu, Junhan Zhao, Jiaying LiuAAAI 2024 · 23 citations
