CVPR Findings2026
The DeepSpeak Dataset
Sarah Barrington, Maty Bohacek, Hany Farid
Abstract
Deepfakes represent a growing concern across domains such as disinformation, fraud, and non-consensual media. In particular, the rise of video conference and identity-driven attacks in high-stakes scenarios-such as impostor hiringdemands new forensic resources. Despite significant efforts to develop robust detection classifiers to distinguish the real from the fake, commonly used training datasets remain inadequate: relying on low-quality and outdated deepfake generators, consisting of content scraped from online repositories without participant consent, lacking in multimodal coverage, and rarely employing identity-matching protocols to ensure realistic fakes. To overcome these limitations, we present the DeepSpeak dataset, a diverse and multimodal dataset comprising over 100 hours of authentic and deepfake audiovisual content, specifically focused on the challenging and diverse "talking heads" context. We contribute: i) more than 50 hours of real, self-recorded data collected from 500 diverse and consenting participants, ii) more than 50 hours of state-of-the-art audio and visual deepfakes generated using 14 video synthesis engines and three voice cloning engines, and iii) an embedding-based, identity-matching approach to ensure the creation of convincing, high-quality identity face swaps that realistically simulate adversarial deepfake attacks. We also perform large-scale evaluations of state-ofthe-art deepfake detectors and show that, without retraining, these detectors fail to generalize to this DeepSpeak dataset, highlighting the importance of a large and diverse dataset containing deepfakes from the latest generative-AI tools.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- X-AVDT: Audio-Visual Cross-Attention for Robust Deepfake Detection cites Learning Transferable Visual Models From Natural Language Supervision
- X-AVDT: Audio-Visual Cross-Attention for Robust Deepfake Detection cites A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
- Robust Speech Recognition via Large-Scale Weak Supervision cites Learning Transferable Visual Models From Natural Language Supervision
Cited by top-tier papers2
Builds on9
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford, Jong Wook Kim, Tao Xu, Greg Brockman et al.ICML 2023 · 6,966 citations
- A Lip Sync Expert Is All You Need for Speech to Lip Generation In the WildK. R. Prajwal, Rudrabha Mukhopadhyay, Vinay P. Namboodiri, C. V. JawaharACM MM 2020 · 869 citations
- Towards Robust Blind Face Restoration with Codebook Lookup TransformerShangchen Zhou, Kelvin C. K. Chan, Chongyi Li, Chen Change LoyNeurIPS 2022 · 431 citations
- SimSwap: An Efficient Framework For High Fidelity Face SwappingRenwang Chen, Xuanhong Chen, Bingbing Ni, Yanhao GeACM MM 2020 · 409 citations
Related papers
- ILLUSION: Unveiling Truth with a Comprehensive Multi-Modal, Multi-Lingual Deepfake DatasetKartik Thakral, Rishabh Ranjan, Akanksha Singh, Akshat Jain et al.ICLR 2025
- SpeechFake: A Large-Scale Multilingual Speech Deepfake Dataset Incorporating Cutting-Edge Generation MethodsWen Huang, Yanmei Gu, Zhiming Wang, Huijia Zhu et al.ACL 2025
- I Can Hear You: Selective Robust Training for Deepfake Audio DetectionZirui Zhang, Wei Hao, Aroon Sankoh, William Lin et al.ICLR 2025
- AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake DatasetZhixi Cai, Shreya Ghosh, Aman Pankaj Adatia, Munawar Hayat et al.ACM MM 2024 · 51 citations
- CoCoVideo: The High-Quality Commercial-Model-Based Contrastive Benchmark for AI-Generated Video DetectionHuidong Feng, Wentao Chen, Jie Chen, Xinqi Cai et al.CVPR 2026 · 2 citations
