StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-Based Generator

and few-shot scenarios. Specifically, we design a maskguided spatial information encoding module that preserves the details of the given face. The mouth shapes are accurately modified by audio through modulated convolutions. Moreover, our design also enables personalized lip-sync by introducing style space and generator refinement on only limited frames. Thus the identity and talking style of a target person could be accurately preserved. Extensive experiments demonstrate the effectiveness of our method in producing high-fidelity results on a variety of scenes. Resources can be found at https://hangz-nju-cuhk. github.io/projects/StyleSync.