EmoTaG: Emotion-Aware Talking Head Synthesis on Gaussian Splatting with Few-Shot Personalization
Haolan Xu, Keli Cheng, Lei Wang, Ning Bi, Xiaoming Liu
Abstract
Audio-driven 3D talking head synthesis has advanced rapidly with Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). Few-shot methods enable instant personalization by reconstructing high-fidelity avatars from only a few seconds of video. However, achieving natural talking-head generation further requires strong emotion-aware motion modeling, and existing few-shot approaches exhibit geometric instability and audio-emotion mismatch under expressive facial motion. In this work, we present EmoTaG, a few-shot emotion-aware 3D talking head synthesis framework built on the Pretrain-and-Adapt paradigm. Our key insight is to reformulate motion prediction in a structured FLAME parameter space rather than directly deforming 3D Gaussians, which introduces strong geometric priors for stable and interpretable motion. Building upon this, we propose a Gated Residual Motion Network (GRMN), which can capture emotional prosody from audio while supplementing head pose and upper-face cues absent in audio to enable expressive yet stable motion generation. Extensive experiments demonstrate that EmoTaG achieves state-of-the-art performance in emotional expressiveness, lip synchronization, visual realism, and motion stability.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers1
Ask how each one uses itBuilds on32
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech RepresentationsAlexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, Michael AuliNeurIPS 2020 · 9,451 citations
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
- Few-Shot Adversarial Learning of Realistic Neural Talking Head ModelsEgor Zakharov, Aliaksandra Shysheya, Egor Burkov, Victor S. LempitskyICCV 2019 · 687 citations
- AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head SynthesisYudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu et al.ICCV 2021 · 510 citations
- AdaFace: Quality Adaptive Margin for Face RecognitionMinchul Kim, Anil K. Jain, Xiaoming LiuCVPR 2022 · 509 citations
Related papers
- AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head SynthesisDongze Li, Kang Zhao, Wei Wang, Bo Peng et al.AAAI 2024 · 25 citations
- GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian SplattingXiaobao Wei, Peng Chen, Guangyu Li, Ming Lu et al.ICCV 2025 · 2 citations
- GaussianTalker: Speaker-specific Talking Head Synthesis via 3D Gaussian SplattingHongyun Yu, Zhan Qu, Qihang Yu, Jianchuan Chen et al.ACM MM 2024 · 28 citations
- InsTaG: Learning Personalized 3D Talking Head from Few-Second VideoJiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng et al.CVPR 2025
- MixedGaussianAvatar: Realistically and Geometrically Accurate Head Avatar via Mixed 2D-3D GaussiansPeng Chen, Xiaobao Wei, Qingpo Wuwu, Xinyi Wang et al.ACM MM 2025 · 2 citations
