High-Fidelity and Freely Controllable Talking Head Video Generation
Yue Gao, Yuan Zhou, Jinglu Wang, Xiao Li, Xiang Ming, Yan Lu
Abstract
PECHead (a). Frontalization Input Yaw Pitch Roll Input Neutral Smile Laugh Input Smile Expressionless Roll (b). Free pose and expression editing Figure 1. Presented herein are representative results showcasing the effectiveness of our proposed method in the tasks of frontalization, as well as free pose and expression editing. (a) FOMM [46] often produces face distortion issues, while our model PECHead generates high-fidelity results. (b) Our proposed framework empowers the generation of talking head videos that offer free control over the head pose and expression. More results can be found in the supplementary materials.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
18 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis cites Few-Shot Adversarial Learning of Realistic Neural Talking Head Models
- VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time cites AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
- VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment cites Few-Shot Adversarial Learning of Realistic Neural Talking Head Models
Cited by top-tier papers18
- VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real TimeSicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang et al.NeurIPS 2024 · 253 citations
- VISTA: A Test-Time Self-Improving Video Generation AgentDo Xuan Long, Xingchen Wan, Hootan Nakhost, Chen-Yu Lee et al.CVPR 2026 · 30 citations
- ShowMaker: Creating High-Fidelity 2D Human Video via Fine-Grained Diffusion ModelingQuanwei Yang, Jiazhi Guan, Kaisiyuan Wang, Lingyun Yu et al.NeurIPS 2024 · 21 citations
- VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head ReenactmentPhong Tran, Egor Zakharov, Long-Nhat Ho, Anh Tuan Tran et al.CVPR 2024 · 15 citations
- FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled AudioChao Xu, Yang Liu, Jiazheng Xing, Weida Wang et al.CVPR 2024 · 11 citations
Builds on21
- Everybody Dance NowCaroline Chan, Shiry Ginosar, Tinghui Zhou, Alexei A. EfrosICCV 2019 · 840 citations
- Few-Shot Adversarial Learning of Realistic Neural Talking Head ModelsEgor Zakharov, Aliaksandra Shysheya, Egor Burkov, Victor S. LempitskyICCV 2019 · 687 citations
- BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and AlignmentKelvin C. K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change LoyCVPR 2022 · 522 citations
- Deep Contextual Video CompressionJiahao Li, Bin Li, Yan LuNeurIPS 2021 · 518 citations
- AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head SynthesisYudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu et al.ICCV 2021 · 510 citations
Related papers
- Context-Aware Talking-Head Video EditingSonglin Yang, Wei Wang, Jun Ling, Bo Peng et al.ACM MM 2023 · 10 citations
- Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head SynthesisDuomin Wang, Yu Deng, Zixin Yin, Heung-Yeung Shum et al.CVPR 2023
- Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual RepresentationHang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy et al.CVPR 2021
- THEval. Evaluation Framework for Talking Head Video GenerationNabyl Quignon, Baptiste Chopin, Yaohui Wang, Antitza DantchevaCVPR Findings 2026
- FG-EmoTalk: Talking Head Video Generation with Fine-Grained Controllable Facial ExpressionsZhaoxu Sun, Yuze Xuan, Fang Liu, Yang XiangAAAI 2024 · 13 citations
