EMNLP Findings2020
Learning Robust and Multilingual Speech Representations
Kazuya Kawakami, Luyu Wang, Chris Dyer, Phil Blunsom, Aäron van den Oord
Abstract
Unsupervised speech representation learning has shown remarkable success at finding representations that correlate with phonetic structures and improve downstream speech recognition performance. However, most research has been focused on evaluating the representations in terms of their ability to improve the performance of speech recognition systems on read English (e.g. Wall Street Journal and Lib-riSpeech). This evaluation methodology overlooks two important desiderata that speech representations should have: robustness to domain shifts and transferability to other languages. In this paper we learn representations from up to 8000 hours of diverse and noisy speech data and evaluate the representations by looking at their robustness to domain shifts and their ability to improve recognition performance in many languages. We find that our representations confer significant robustness advantages to the resulting recognition systems: we see significant improvements in out-of-domain transfer relative to baseline feature sets and the features likewise provide improvements in 25 phonetically diverse languages.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
6 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Representation Learning for Sequence Data with Deep Autoencoding Predictive Components cites wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
- UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data cites vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
- Unsupervised Speech Recognition cites vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
- Unsupervised Speech Recognition cites wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations cites Data-Efficient Image Recognition with Contrastive Predictive Coding
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations cites vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
Cited by top-tier papers6
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech RepresentationsAlexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, Michael AuliNeurIPS 2020 · 9,451 citations
- Unsupervised Speech RecognitionAlexei Baevski, Wei-Ning Hsu, Alexis Conneau, Michael AuliNeurIPS 2021 · 309 citations
- UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled DataChengyi Wang, Yu Wu, Yao Qian, Ken'ichi Kumatani et al.ICML 2021 · 140 citations
- u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled ModalityWei-Ning Hsu, Bowen ShiNeurIPS 2022 · 58 citations
- Representation Learning for Sequence Data with Deep Autoencoding Predictive ComponentsJunwen Bai, Weiran Wang, Yingbo Zhou, Caiming XiongICLR 2021 · 1 citation
Builds on2
Related papers
- VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and InterpretationChanghan Wang, Morgane Rivière, Ann Lee, Anne Wu et al.ACL 2021
- Cross-Lingual Transfer with Class-Weighted Language-Invariant RepresentationsRuicheng Xian, Heng Ji, Han ZhaoICLR 2022 · 5 citations
- Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford, Jong Wook Kim, Tao Xu, Greg Brockman et al.ICML 2023 · 6,966 citations
- MauBERT: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units DiscoveryAngelo Ortiz Tandazo, Manel Khentout, Youssef Benchekroun, Thomas Hueber et al.ACL 2026 · 1 citation
- Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation ModelsYuchen Hu, Chen Chen, Chao-Han Huck Yang, Chengwei Qin et al.NeurIPS 2024 · 14 citations
