Lune

ICLR2026顶会

MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Control

Sahil Kumar, Namrataben Patel, Honggang Wang, Youshan Zhang

摘要

MambaVoiceCloning (MVC) asks whether the conditioning path of diffusion-based TTS can be made fully SSM-only at inference—removing all attention and explicit RNN-style recurrence layers across text, rhythm, and prosody—while preserving or improving quality under controlled conditions. MVC combines a gated bidirectional Mamba text encoder, a Temporal Bi-Mamba supervised by a lightweight alignment teacher discarded after training, and an Expressive Mamba with AdaLN modulation, yielding linear-time O(T)\mathcal{O}(T) conditioning with bounded activation memory and practical finite look-ahead streaming. Unlike prior Mamba--TTS systems that remain hybrid at inference, MVC removes attention-based duration and style modules under a fixed StyleTTS2 mel--diffusion--vocoder backbone. Trained on LJSpeech/LibriTTS and evaluated on VCTK, CSS10 (ES/DE/FR), and long-form Gutenberg passages, MVC achieves modest but statistically reliable gains over StyleTTS2, VITS, and Mamba--attention hybrids in MOS/CMOS, F0_0 RMSE, MCD, and WER, while reducing encoder parameters to 21M and improving throughput by 1.6×1.6\times. Diffusion remains the dominant latency source, but SSM-only conditioning improves memory footprint, stability, and deployability. Code available at: https://github.com/sahilkumar15/MVC.

问 Lune

几分钟读透这篇 Paper,不用再花一下午。

Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。

可以从这些问题问起

Lune 可以查阅

全文 · 相关的 Paper

获取答案

免费开始,无需绑卡

脉络

  • StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models 引用了 Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech
  • StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models 引用了 ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-Speech
  • StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models 引用了 Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech
  • ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-Speech 引用了 Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech
  • ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-Speech 引用了 FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
  • ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-Speech 引用了 Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech
  • Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech 引用了 FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
它引用的引用它的从左到右按年份排列,引用越多,位置越高。

它引用的 Paper6

  1. Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-SpeechJaehyeon Kim, Jungil Kong, Juhee SonICML 2021 · 1,267 次引用
  2. Grad-TTS: A Diffusion Probabilistic Model for Text-to-SpeechVadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova 等ICML 2021 · 715 次引用
  3. FastSpeech 2: Fast and High-Quality End-to-End Text to SpeechYi Ren, Chenxu Hu, Xu Tan, Tao Qin 等ICLR 2021 · 513 次引用
  4. StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language ModelsYinghao Aaron Li, Cong Han, Vinay S. Raghavan, Gavin Mischler 等NeurIPS 2023 · 324 次引用
  5. ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-SpeechRongjie Huang, Zhou Zhao, Huadai Liu, Jinglin Liu 等ACM MM 2022 · 182 次引用

相关 Paper

  1. DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech SynthesisYinghao Aaron Li, Rithesh Kumar, Zeyu JinICML 2025
  2. DiffuMamba: High-Throughput Diffusion LMs with Mamba BackboneVaibhav Singh, Oleksiy Ostapenko, Pierre-André Noël, Eugene Belilovsky 等ICML 2026
  3. Mamba-3: Improved Sequence Modeling using State Space PrinciplesAakash Sunil Lahoti, Kevin Y. Li, Berlin Chen, Caitlin Wang 等ICLR 2026 · 96 次引用
  4. DisCo_Speech: Controllable Zero-Shot Speech Generation with A Disentangled Speech CodecTao Li, Wenshuo Ge, Zhichao Wang, Zihao Cui 等ACL 2026 · 1 次引用
  5. StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow MatchingJixun Yao, Yuguang Yang, Yu Pan, Ziqian Ning 等AAAI 2025 · 13 次引用
黄昏的海面,两侧是细线勾勒的悬崖