Exploring Timeline Control for Facial Motion Generation

Brow Brow Up Eye Close Widen Squint Mouth Smile Gaze Gaze right Pose Head Left&Down Head Right Head Left (Rendered) Facial motions aligned with timelines User-specified facial motion timelines Time … Brow Brow Down Brow Up Eye Squint Widen Mouth Smile Gaze Gaze right Pose Head Right Natural facial motion sequences Facial motion timelines from annotation Time … Annotate 2 4 (sec) 0 2 4 (sec) 0 Figure 1. We introduce a new control signal for facial motion generation: timeline control. We first utilize a labor-efficient approach to annotate the time intervals of facial motion at a frame-level granularity. Using the annotations, we propose a model that can generate natural facial motions aligned with an input timeline. Compared to previous controls like audio and text, timeline control enables precise temporal control of facial motions. In this paper, facial motions are rendered into photorealistic videos for better visualization.