Learning Repeatable Speech Embeddings Using An Intra-class Correlation Regularizer
Jianwei Zhang, Suren Jayasuriya, Visar Berisha
Abstract
A good supervised embedding for a specific machine learning task is only sensitive to changes in the label of interest and is invariant to other confounding factors. We leverage the concept of repeatability from measurement theory to describe this property and propose to use the intra-class correlation coefficient (ICC) to evaluate the repeatability of embeddings. We then propose a novel regularizer, the ICC regularizer, as a complementary component for contrastive losses to guide deep neural networks to produce embeddings with higher repeatability. We use simulated data to explain why the ICC regularizer works better on minimizing the intra-class variance than the contrastive loss alone. We implement the ICC regularizer and apply it to three speech tasks: speaker verification, voice style conversion, and a clinical application for detecting dysphonic voice. The experimental results demonstrate that adding an ICC regularizer can improve the repeatability of learned embeddings compared to only using the contrastive loss; further, these embeddings lead to improved performance in these downstream tasks.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning cites A Simple Framework for Contrastive Learning of Visual Representations
- VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning cites What Makes for Good Views for Contrastive Learning?
- Supervised Contrastive Learning cites A Simple Framework for Contrastive Learning of Visual Representations
- Supervised Contrastive Learning cites What Makes for Good Views for Contrastive Learning?
- What Makes for Good Views for Contrastive Learning? cites A Simple Framework for Contrastive Learning of Visual Representations
Cited by top-tier papers1
Builds on9
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- Supervised Contrastive LearningPrannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna et al.NeurIPS 2020 · 7,049 citations
- What Makes for Good Views for Contrastive Learning?Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan et al.NeurIPS 2020 · 1,631 citations
- VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised LearningAdrien Bardes, Jean Ponce, Yann LeCunICLR 2022 · 1,226 citations
- A Theoretical Analysis of the Number of Shots in Few-Shot LearningTianshi Cao, Marc T. Law, Sanja FidlerICLR 2020 · 75 citations
Related papers
- Seeing Your Speech Style: A Novel Zero-Shot Identity-Disentanglement Face-based Voice ConversionYan Rong, Li LiuAAAI 2025 · 11 citations
- Learning Invariant Representations using Inverse Contrastive LossAditya Kumar Akash, Vishnu Suresh Lokhande, Sathya N. Ravi, Vikas SinghAAAI 2021 · 10 citations
- Improving Zero-Shot Voice Style Transfer via Disentangled Representation LearningSiyang Yuan, Pengyu Cheng, Ruiyi Zhang, Weituo Hao et al.ICLR 2021 · 64 citations
- Improving Transformation Invariance in Contrastive Representation LearningAdam Foster, Rattana Pukdee, Tom RainforthICLR 2021 · 25 citations
- An Information Theory Perspective on Variance-Invariance-Covariance RegularizationRavid Shwartz-Ziv, Randall Balestriero, Kenji Kawaguchi, Tim G. J. Rudner et al.NeurIPS 2023 · 21 citations
