Lune

CVPR Findings2026

TalkVid: A Large-Scale Diversified Dataset for Audio-Driven Talking Head Synthesis

Shunian Chen, Hejin Huang, Yexin Liu, Zihan Ye, Pengcheng Chen, Chenghao Zhu, Michael Guan, Rongsheng Wang, Junying Chen, Jianye Hou, Bo Li, Guanbin Li, Ser-Nam Lim, Harry Yang, Benyou Wang

Abstract

Audio-driven talking head synthesis has achieved remarkable photorealism, yet state-of-the-art (SOTA) models exhibit a critical failure: they lack generalization to the full spectrum of human diversity in ethnicity, language, and age-groups. We argue this generalization gap is a direct symptom of limitations in existing training data, which lack the necessary scale, quality, and diversity. To address this challenge, we introduce TalkVid, a new largescale, high-quality, and diverse dataset containing 1244 hours of video from 7729 unique speakers. TalkVid is curated through a principled, multi-stage automated pipeline that filters for motion stability, aesthetic quality, and facial detail, and is validated against human judgments to ensure its reliability. Furthermore, we construct and release TalkVid-Bench, a stratified evaluation set of 500 clips that is balanced across key demographic and linguistic axes. Our experiments demonstrate that a model trained on TalkVid outperforms counterparts trained on previous datasets, exhibiting superior cross-dataset generalization. Crucially, our analysis on TalkVid-Bench reveals performance disparities across subgroups that are obscured by traditional aggregate metrics, underscoring its necessity for future research. Code and data can be found in https:// github.com/FreedomIntelligence/TalkVid

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset cites A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
  • Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer cites Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset
  • Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer cites VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time
  • Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer cites A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
  • VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time cites Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset
  • VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time cites A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
  • Just-Dub-It: Video dubbing via Joint Audio-Visual Diffusion cites Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset
  • Just-Dub-It: Video dubbing via Joint Audio-Visual Diffusion cites A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
  • Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection cites Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset
  • Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection cites Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake DetectionTianxiao Li, Zhenglin Huang, Haiquan Wen, Yiwei He et al.CVPR 2026 · 5 citations
  2. Just-Dub-It: Video dubbing via Joint Audio-Visual DiffusionAnthony Chen, Naomi Ken Korem, Tavi Halperin, Matan Ben-Yosef et al.SIGGRAPH 2026 · 2 citations

Builds on5

  1. A Lip Sync Expert Is All You Need for Speech to Lip Generation In the WildK. R. Prajwal, Rudrabha Mukhopadhyay, Vinay P. Namboodiri, C. V. JawaharACM MM 2020 · 869 citations
  2. Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical PerspectivesHaoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen et al.ICCV 2023 · 371 citations
  3. VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real TimeSicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang et al.NeurIPS 2024 · 253 citations
  4. Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual DatasetZhimeng Zhang, Lincheng Li, Yu Ding, Changjie FanCVPR 2021
  5. Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion TransformerJiahao Cui, Hui Li, Yun Zhan, Hanlin Shang et al.CVPR 2025

Related papers

  1. DH-FaceVid-1K: A Large-Scale High-Quality Dataset for Face Video GenerationDonglin Di, He Feng, Wenzhang Sun, Yongjia Ma et al.ICCV 2025 · 1 citation
  2. SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human GenerationYouliang Zhang, Zhaoyang Li, Duomin Wang, jiahe zhang et al.ICLR 2026 · 30 citations
  3. OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video GenerationHui Li, Mingwang Xu, Yun Zhan, Shan Mu et al.CVPR 2025
  4. The DeepSpeak DatasetSarah Barrington, Maty Bohacek, Hany FaridCVPR Findings 2026
  5. Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion ModelFei Shen, Cong Wang, Junyao Gao, Qin Guo et al.ICML 2025
Dusk over the sea between two cliffs drawn in fine vertical lines