Lune

SIGGRAPH2024Top-tier venue

Filter-Guided Diffusion for Controllable Image Generation

Zeqi Gu, Ethan Yang, Abe Davis

Abstract

3 σ value σ spatial σ value Generation with Guidance from Corresponding Bilateral Parameters Decoded Guide Structure for Different Bilateral Parameters Sampling with Different FGD Parameters Yields Higher-Quality Results in Less Time Than it Takes Previous SOTA Methods to Generate One Output Guidance Strength Figure 1: We present Filter-Guided Diffusion (FGD), a training-free black box method for conditioning image diffusion on the structure of example images. FGD works by adding a fast filtering step between each iteration of the diffusion process. Building on classic image processing theory, we can design this filtering step to preserve the structure of a given guide image. The grid on the far left shows the effects of classic bilateral filters with different parameters on an image. The middle grid shows the result of using the same filters for guidance with FGD (in this case, using an empty prompt). The spatial filter size, which controls the scale of blur in a traditional bilateral, determines the spatial scale at which the network may vary from structure of the input image. The value filter size, which controls edge preservation in the traditional bilateral, determines how much the diffusion process should respect the edges of the guide. In addition to the filter's parameters, we can control the overall strength of guidance to determine how closely the result should resemble the input. The right shows two examples of image-to-image translation with varying guidance strength. Results become closer to the guide image as guidance strength increases to the right.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • High-Resolution Image Synthesis with Latent Diffusion Models cites Denoising Diffusion Probabilistic Models
  • Generative Photomontage cites High-Resolution Image Synthesis with Latent Diffusion Models
  • Generative Photomontage cites Denoising Diffusion Probabilistic Models
  • Directly Denoising Diffusion Models cites High-Resolution Image Synthesis with Latent Diffusion Models
  • Directly Denoising Diffusion Models cites Denoising Diffusion Probabilistic Models
  • Learning Transferable Visual Models From Natural Language Supervision cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. AnchorHOI: Zero-shot Generation of 4D Human-Object Interaction via Anchor-based Prior DistillationSisi Dai, Kai XuAAAI 2026
  2. Generative PhotomontageSean J. Liu, Nupur Kumari, Ariel Shamir, Jun-Yan ZhuCVPR 2025

Builds on17

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  3. Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
  4. High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
  5. Directly Denoising Diffusion ModelsDan Zhang, Jingjing Wang, Feng LuoICML 2024 · 11,724 citations

Related papers

  1. Frequency-Guided Diffusion for Training-Free Text-Driven Image TranslationZheng Gao, Jifei Song, Zhensong Zhang, Jiankang Deng et al.ICCV 2025 · 1 citation
  2. Guiding a Diffusion Model with a Bad Version of ItselfTero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen et al.NeurIPS 2024 · 338 citations
  3. On Distillation of Guided Diffusion ModelsChenlin Meng, Robin Rombach, Ruiqi Gao, Diederik P. Kingma et al.CVPR 2023
  4. Manifold Preserving Guided DiffusionYutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida et al.ICLR 2024 · 148 citations
  5. Frequency-Controlled Diffusion Model for Versatile Text-Guided Image-to-Image TranslationXiang Gao, Zhengbo Xu, Junhan Zhao, Jiaying LiuAAAI 2024 · 23 citations
Dusk over the sea between two cliffs drawn in fine vertical lines