Lune

EMNLP Findings2023

Non-parallel Accent Transfer based on Fine-grained Controllable Accent Modelling

Linqin Wang, Zhengtao Yu, Yuanzhang Yang, Shengxiang Gao, Cunli Mao, Yuxin Huang

Abstract

Existing accent transfer works rely on parallel data or speech recognition models. This paper focuses on the practical application of accent transfer and aims to implement accent transfer using non-parallel datasets. The study has encountered the challenge of speech representation disentanglement and modeling accents. In our accent modeling transfer framework, we manage to solve these problems by two proposed methods. First, we learn the suprasegmental information associated with tone to finely model the accents in terms of tone and rhythm. Second, we propose to use mutual information learning to disentangle the accent features and control the accent of the generated speech during the inference time. Experiments show that the proposed framework attains superior performance to the baseline models in terms of accentedness and audio quality.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on3

  1. HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech SynthesisJungil Kong, Jaehyeon Kim, Jaekyoung BaeNeurIPS 2020 · 2,890 citations
  2. CLUB: A Contrastive Log-ratio Upper Bound of Mutual InformationPengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu et al.ICML 2020 · 512 citations
  3. Unsupervised Speech Decomposition via Triple Information BottleneckKaizhi Qian, Yang Zhang, Shiyu Chang, Mark Hasegawa-Johnson et al.ICML 2020 · 210 citations

Related papers

  1. Improving Zero-Shot Voice Style Transfer via Disentangled Representation LearningSiyang Yuan, Pengyu Cheng, Ruiyi Zhang, Weituo Hao et al.ICLR 2021 · 64 citations
  2. Global Prosody Style Transfer Without Text TranscriptionsKaizhi Qian, Yang Zhang, Shiyu Chang, Jinjun Xiong et al.ICML 2021 · 25 citations
  3. INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech RecognitionEunseop Yoon, Hee Suk Yoon, John B. Harvill, Mark Hasegawa-Johnson et al.ACL Findings 2023 · 3 citations
  4. Convert and Speak: Zero-shot Accent Conversion with Minimum SupervisionZhijun Jia, Huaying Xue, Xiulian Peng, Yan LuACM MM 2024 · 3 citations
  5. CCSRD: Content-Centric Speech Representation Disentanglement Learning for End-to-End Speech TranslationXiaohu Zhao, Haoran Sun, Yikun Lei, Shaolin Zhu et al.EMNLP Findings 2023 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines