Lune

ICML2026Top-tier venue

Alethia: a Foundational Encoder for Voice Deepfakes

Yi Zhu, Brahmi Dwivedi, Jayaram Raghuram, Surya Koppisetti

Abstract

Existing voice deepfake detection and localization models rely heavily on representations extracted from speech foundation models (SFMs). However, downstream finetuning has now reached a state of diminishing returns. In this paper, we shift the focus to pretraining and propose a novel recipe that combines bottleneck masked embedding prediction with flow-matching based spectrogram reconstruction . The outcome, Alethia , is the first foundational audio encoder for various voice deepfake detection and localization tasks. We evaluate on 5 different tasks with 56 benchmark datasets, and note Alethia significantly outperforms state-of-the-art SFMs with superior robustness to real-world perturbations and zero-shot generalization to unseen domains (e.g., singing deepfakes). We also demonstrate the limitation of discrete targets in masked token prediction, and show the importance of continuous embedding prediction and generative pretraining for capturing deepfake artifacts.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Generative Pre-training for Speech with Flow Matching cites Flow Matching for Generative Modeling
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on8

  1. Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and LanguageAlexei Baevski, Arun Babu, Wei-Ning Hsu, Michael AuliICML 2023 · 137 citations
  2. Flow Matching for Generative ModelingYaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel et al.ICLR 2023 · 87 citations
  3. Generative Pre-training for Speech with Flow MatchingAlexander H. Liu, Matthew Le, Apoorv Vyas, Bowen Shi et al.ICLR 2024 · 66 citations
  4. Transferring Audio Deepfake Detection Capability across LanguagesZhongjie Ba, Qing Wen, Peng Cheng, Yuwei Wang et al.WWW 2023 · 34 citations
  5. Trident of Poseidon: A Generalized Approach for Detecting Deepfake VoicesThien-Phuc Doan, Hung Dinh-Xuan, Taewon Ryu, Inho Kim et al.CCS 2024 · 5 citations

Related papers

  1. Scaling Behavior in Model Fine-tuning for Audio DeepFake DetectionXiang Li, Pin-Yu Chen, Wenqi WeiICML 2026
  2. Investigating Self-Supervised Representations for Audio-Visual Deepfake DetectionDragos-Alexandru Boldisor, Stefan Smeu, Dan Oneata, Elisabeta OneataCVPR 2026 · 2 citations
  3. Investigating Prosodic Signatures via Speech Pre-Trained Models for Audio Deepfake Source AttributionOrchid Chetia Phukan, Drishti Singh, Swarup Ranjan Behera, Arun Balaji Buduru et al.ACL Findings 2025 · 2 citations
  4. SLIM: Style-Linguistics Mismatch Model for Generalized Audio Deepfake DetectionYi Zhu, Surya Koppisetti, Trang Tran, Gaurav BharajNeurIPS 2024 · 41 citations
  5. Phoneme-Level Feature Discrepancies: A Key to Detecting Sophisticated Speech DeepfakesKuiyuan Zhang, Zhongyun Hua, Rushi Lan, Yushu Zhang et al.AAAI 2025 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines