Playmate: Flexible Control of Portrait Animation via 3D-Implicit Space Guided Diffusion
Xingpei Ma, Jiaran Cai, Yuansheng Guan, Shenneng Huang, Qiang Zhang, Shunsi Zhang
Abstract
Figure 1. Playmate can generate lifelike talking faces for arbitrary identity, guided by a speech audio clip and a variety of optional control conditions. (a) shows the generation results under different emotional conditions using the same audio clip. The top row in (b) shows the driving images, while the bottom row shows the generated results. The poses in the generated results are controlled by the driving images, and the lip movements are guided by the driving audio. (c) demonstrates highly accurate lip synchronization and vivid, rich expressions across different style images.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 475d744b-03f4-4ac2-b149-222596e6d6deCited by top-tier papers3
- AUHead: Realistic Emotional Talking Head Generation via Action Units ControlJiayi Lyu, Leigang Qu, Wenjing Zhang, Hanyu Jiang et al.ICLR 2026 · 2 citations
- SyncDreamer: Controllable and Expressive Avatar Generation Beyond the Talking HeadFatemeh Nazarieh, Zhenhua Feng, Diptesh Kanojia, Josef Kittler et al.CVPR 2026
- Training-Free Multi-Character Audio-Driven Animation via Diffusion Transformer with Reward FeedbackXingpei Ma, Shenneng Huang, Jiaran Cai, Yuansheng Guan et al.AAAI 2026
Builds on25
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Denoising Diffusion Implicit ModelsJiaming Song, Chenlin Meng, Stefano ErmonICLR 2021 · 11,743 citations
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech RepresentationsAlexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, Michael AuliNeurIPS 2020 · 9,451 citations
Related papers
- Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual RepresentationHang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy et al.CVPR 2021
- Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head SynthesisDuomin Wang, Yu Deng, Zixin Yin, Heung-Yeung Shum et al.CVPR 2023
- PC-Talk: Precise Facial Animation Control for Audio-Driven Talking Face GenerationBaiqin Wang, Xiangyu Zhu, Fan Shen, Hao Xu et al.CVPR 2026 · 8 citations
- That's What I Said: Fully-Controllable Talking Face GenerationYoungjoon Jang, Kyeongha Rho, Jong-Bin Woo, Hyeongkeun Lee et al.ACM MM 2023 · 7 citations
- Talking Face Generation with Expression-Tailored Generative Adversarial NetworkDan Zeng, Han Liu, Hui Lin, Shiming GeACM MM 2020 · 30 citations
