Lune

ACM MM2020Top-tier venue

DeepFacePencil: Creating Face Images from Freehand Sketches

Yuhang Li, Xuejin Chen, Binxin Yang, Zihan Chen, Zhihua Cheng, Zheng-Jun Zha

Abstract

In this paper, we explore the task of generating photo-realistic face images from hand-drawn sketches. Existing image-to-image translation methods require a large-scale dataset of paired sketches and images for supervision. They typically utilize synthesized edge maps of face images as training data. However, these synthesized edge maps strictly align with the edges of the corresponding face images, which limit their generalization ability to real hand-drawn sketches with vast stroke diversity. To address this problem, we propose DeepFacePencil, an effective tool that is able to generate photo-realistic face images from hand-drawn sketches, based on a novel dual generator image translation network during training. A novel spatial attention pooling (SAP) is designed to adaptively handle stroke distortions which are spatially varying to support various stroke styles and different level of details. We conduct extensive experiments and the results demonstrate the superiority of our model over existing methods on both image quality and model generalization to hand-drawn sketches.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

8 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • FENeRF: Face Editing in Neural Radiance Fields cites MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
  • MaskGAN: Towards Diverse and Interactive Facial Image Manipulation cites SC-FEGAN: Face Editing Generative Adversarial Network With User's Sketch and Color
  • DeepFaceEditing: deep face generation and editing with disentangled geometry and appearance control cites MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
  • DeepFaceEditing: deep face generation and editing with disentangled geometry and appearance control cites SC-FEGAN: Face Editing Generative Adversarial Network With User's Sketch and Color
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers8

  1. Deep Graph-neighbor Coherence Preserving Network for Unsupervised Cross-modal HashingJun Yu, Hao Zhou, Yibing Zhan, Dacheng TaoAAAI 2021 · 184 citations
  2. FENeRF: Face Editing in Neural Radiance FieldsJingxiang Sun, Xuan Wang, Yong Zhang, Xiaoyu Li et al.CVPR 2022 · 121 citations
  3. DeepFaceEditing: deep face generation and editing with disentangled geometry and appearance controlShu-Yu Chen, Feng-Lin Liu, Yu-Kun Lai, Paul L. Rosin et al.SIGGRAPH 2021 · 35 citations
  4. Text as Neural Operator: Image Manipulation by Text InstructionTianhao Zhang, Hung-Yu Tseng, Lu Jiang, Weilong Yang et al.ACM MM 2021 · 28 citations
  5. AniFaceDrawing: Anime Portrait Exploration during Your SketchingZhengyu Huang, Haoran Xie, Tsukasa Fukusato, Kazunori MiyataSIGGRAPH 2023 · 24 citations

Builds on3

  1. SC-FEGAN: Face Editing Generative Adversarial Network With User's Sketch and ColorYoungjoo Jo, Jongyoul ParkICCV 2019 · 325 citations
  2. Real-World Person Re-Identification via Degradation Invariance LearningYukun Huang, Zheng-Jun Zha, Xueyang Fu, Richang Hong et al.CVPR 2020
  3. MaskGAN: Towards Diverse and Interactive Facial Image ManipulationCheng-Han Lee, Ziwei Liu, Lingyun Wu, Ping LuoCVPR 2020

Related papers

  1. DeepFaceDrawing: deep generation of face images from sketchesShu-Yu Chen, Wanchao Su, Lin Gao, Shihong Xia et al.SIGGRAPH 2020 · 145 citations
  2. Controllable Face Sketch-Photo Synthesis with Flexible Generative PriorsKun Cheng, Mingrui Zhu, Nannan Wang, Guozhang Li et al.ACM MM 2023 · 5 citations
  3. Stroke2Sketch: Harnessing Stroke Attributes for Training-Free Sketch GenerationRui Yang, Huining Li, Yiyi Long, Xiaojun Wu et al.ICCV 2025 · 2 citations
  4. DeSmoothGAN: Recovering Details of Smoothed Images via Spatial Feature-wise Transformation and Full AttentionYifei Huang, Chenhui Li, Xiaohu Guo, Jing Liao et al.ACM MM 2020 · 4 citations
  5. Picture that Sketch: Photorealistic Image Generation from Abstract SketchesSubhadeep Koley, Ayan Kumar Bhunia, Aneeshan Sain, Pinaki Nath Chowdhury et al.CVPR 2023
Dusk over the sea between two cliffs drawn in fine vertical lines