DemoCaricature: Democratising Caricature Generation with a Rough Sketch
Dar-Yen Chen, Ayan Kumar Bhunia, Subhadeep Koley, Aneeshan Sain, Pinaki Nath Chowdhury, Yi-Zhe Song
Abstract
In this paper, we democratise caricature generation, empowering individuals to effortlessly craft personalised caricatures with just a photo and a conceptual sketch. Our objective is to strike a delicate balance between abstraction and identity, while preserving the creativity and subjectivity inherent in a sketch. To achieve this, we present Explicit Rank-1 Model Editing alongside single-image personalisation, selectively applying nuanced edits to cross-attention layers for a seamless merge of identity and style. Additionally, we propose Random Mask Reconstruction to enhance robustness, directing the model to focus on distinctive identity and style features. Crucially, our aim is not to replace artists but to eliminate accessibility barriers, allowing enthusiasts to engage in the artistry.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers8
- HarmonyCut: Supporting Creative Chinese Paper-cutting Design with Form and Connotation HarmonyHuanchen Wang, Tianrun Qiu, Jiaping Li, Zhicong Lu et al.CHI 2025 · 17 citations
- S2TD-Face: Reconstruct a Detailed 3D Face with Controllable Texture from a Single SketchZidu Wang, Xiangyu Zhu, Jiang Yu, Tianshuo Zhang et al.ACM MM 2024 · 3 citations
- DynamicID: Zero-Shot Multi-ID Image Personalization With Flexible Facial EditabilityXirui Hu, Jiahao Wang, Hao Chen, Weizhan Zhang et al.ICCV 2025 · 3 citations
- CaricHarmony: Contrastive Diffusion Paths for Identity-Preserving Caricature SynthesisDongyu Wang, Dar-Yen Chen, Yi-Zhe SongCVPR 2026
- Text-to-Image Diffusion Models are Great Sketch-Photo MatchmakersSubhadeep Koley, Ayan Kumar Bhunia, Aneeshan Sain, Pinaki Nath Chowdhury et al.CVPR 2024
Builds on35
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
Related papers
- CaricatureBooth: Data-Free Interactive Caricature Generation in a Photo BoothZhiyu Qu, Yunqi Miao, Zhensong Zhang, Jifei Song et al.CVPR 2025
- CariPainter: Sketch Guided Interactive Caricature GenerationXin Huang, Dong Liang, Hongrui Cai, Juyong Zhang et al.ACM MM 2022 · 5 citations
- StyleCariGAN: caricature generation via StyleGAN feature map modulationWonjong Jang, Gwangjin Ju, Yucheol Jung, Jiaolong Yang et al.SIGGRAPH 2021 · 51 citations
- Controllable Face Sketch-Photo Synthesis with Flexible Generative PriorsKun Cheng, Mingrui Zhu, Nannan Wang, Guozhang Li et al.ACM MM 2023 · 5 citations
- Modeling Caricature Expressions by 3D Blendshape and Dynamic TextureKeyu Chen, Jianmin Zheng, Jianfei Cai, Juyong ZhangACM MM 2020 · 8 citations
