CVPR Findings2026
TalkVid: A Large-Scale Diversified Dataset for Audio-Driven Talking Head Synthesis
Shunian Chen, Hejin Huang, Yexin Liu, Zihan Ye, Pengcheng Chen, Chenghao Zhu, Michael Guan, Rongsheng Wang, Junying Chen, Jianye Hou, Bo Li, Guanbin Li, Ser-Nam Lim, Harry Yang, Benyou Wang
Abstract
Audio-driven talking head synthesis has achieved remarkable photorealism, yet state-of-the-art (SOTA) models exhibit a critical failure: they lack generalization to the full spectrum of human diversity in ethnicity, language, and age-groups. We argue this generalization gap is a direct symptom of limitations in existing training data, which lack the necessary scale, quality, and diversity. To address this challenge, we introduce TalkVid, a new largescale, high-quality, and diverse dataset containing 1244 hours of video from 7729 unique speakers. TalkVid is curated through a principled, multi-stage automated pipeline that filters for motion stability, aesthetic quality, and facial detail, and is validated against human judgments to ensure its reliability. Furthermore, we construct and release TalkVid-Bench, a stratified evaluation set of 500 clips that is balanced across key demographic and linguistic axes. Our experiments demonstrate that a model trained on TalkVid outperforms counterparts trained on previous datasets, exhibiting superior cross-dataset generalization. Crucially, our analysis on TalkVid-Bench reveals performance disparities across subgroups that are obscured by traditional aggregate metrics, underscoring its necessity for future research. Code and data can be found in https:// github.com/FreedomIntelligence/TalkVid
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset cites A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
- Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer cites Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset
- Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer cites VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time
- Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer cites A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
- VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time cites Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset
- VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time cites A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
- Just-Dub-It: Video dubbing via Joint Audio-Visual Diffusion cites Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset
- Just-Dub-It: Video dubbing via Joint Audio-Visual Diffusion cites A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
- Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection cites Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset
- Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection cites Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer
Cited by top-tier papers2
- Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake DetectionTianxiao Li, Zhenglin Huang, Haiquan Wen, Yiwei He et al.CVPR 2026 · 5 citations
- Just-Dub-It: Video dubbing via Joint Audio-Visual DiffusionAnthony Chen, Naomi Ken Korem, Tavi Halperin, Matan Ben-Yosef et al.SIGGRAPH 2026 · 2 citations
Builds on5
- A Lip Sync Expert Is All You Need for Speech to Lip Generation In the WildK. R. Prajwal, Rudrabha Mukhopadhyay, Vinay P. Namboodiri, C. V. JawaharACM MM 2020 · 869 citations
- Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical PerspectivesHaoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen et al.ICCV 2023 · 371 citations
- VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real TimeSicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang et al.NeurIPS 2024 · 253 citations
- Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual DatasetZhimeng Zhang, Lincheng Li, Yu Ding, Changjie FanCVPR 2021
- Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion TransformerJiahao Cui, Hui Li, Yun Zhan, Hanlin Shang et al.CVPR 2025
Related papers
- DH-FaceVid-1K: A Large-Scale High-Quality Dataset for Face Video GenerationDonglin Di, He Feng, Wenzhang Sun, Yongjia Ma et al.ICCV 2025 · 1 citation
- SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human GenerationYouliang Zhang, Zhaoyang Li, Duomin Wang, jiahe zhang et al.ICLR 2026 · 30 citations
- OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video GenerationHui Li, Mingwang Xu, Yun Zhan, Shan Mu et al.CVPR 2025
- The DeepSpeak DatasetSarah Barrington, Maty Bohacek, Hany FaridCVPR Findings 2026
- Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion ModelFei Shen, Cong Wang, Junyao Gao, Qin Guo et al.ICML 2025
