It's All About Your Sketch: Democratising Sketch Control in Diffusion Models
Subhadeep Koley, Ayan Kumar Bhunia, Deeptanshu Sekhri, Aneeshan Sain, Pinaki Nath Chowdhury, Tao Xiang, Yi-Zhe Song
Abstract
Sketch ControlNet T2I-Adapter Ours Sketch ControlNet T2I-Adapter Ours Edgemap ControlNet T2I-Adapter Ours Our Results Figure 1. Top-left: Comparison of images generated by our method with SGDM [81], ControlNet [90], and T2I-Adapter [55]. Top-right: A set of photos generated by our method. Bottom: While existing methods [55, 90] generate realistic images from pixel-perfect edgemaps, they perform sub-optimally for freehand abstract sketches. (Best view when zoomed in.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 68a683f6-36cf-4643-afff-329062be8b1cCited by top-tier papers18
- AirSketch: Generative Motion to SketchHui Xian Grace Lim, Xuanming Cui, Yogesh S. Rawat, Ser Nam LimNeurIPS 2024 · 4 citations
- SketchingReality: From Freehand Scene Sketches to Photorealistic ImagesAhmed Bourouis, Mikhail Bessmeltsev, Yulia GryaditskayaICLR 2026 · 3 citations
- LOTS of Fashion! Multi-Conditioning for Image Generation via Sketch-Text PairingFederico Girella, Davide Talon, Ziyue Liu, Zanxi Ruan et al.ICCV 2025 · 2 citations
- Universal Multi-Domain Translation via Diffusion RoutersDuc Kieu, Kien Do, Tuan Hoang, Thao Minh Le et al.ICLR 2026 · 2 citations
- Component-Wise Sketching and Generation for Car Interior DesignJeongche Yoon, Seung-Jun Lee, Donghyeok Ma, Seok-Hyung BaeCHI 2026 · 1 citation
Builds on52
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li et al.NeurIPS 2022 · 8,965 citations
Related papers
- Picture that Sketch: Photorealistic Image Generation from Abstract SketchesSubhadeep Koley, Ayan Kumar Bhunia, Aneeshan Sain, Pinaki Nath Chowdhury et al.CVPR 2023
- Block and Detail: Scaffolding Sketch-to-Image GenerationVishnu Sarukkai, Lu Yuan, Mia Tang, Maneesh Agrawala et al.UIST 2024 · 23 citations
- SketchyCOCO: Image Generation From Freehand Scene SketchesChengying Gao, Qi Liu, Qi Xu, Limin Wang et al.CVPR 2020
- DeFLOCNet: Deep Image Editing via Flexible Low-Level ControlsHongyu Liu, Ziyu Wan, Wei Huang, Yibing Song et al.CVPR 2021
- Neural Contours: Learning to Draw Lines From 3D ShapesDifan Liu, Mohamed Nabail, Aaron Hertzmann, Evangelos KalogerakisCVPR 2020
