Lune

ACL Findings2023

INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech Recognition

Eunseop Yoon, Hee Suk Yoon, John B. Harvill, Mark Hasegawa-Johnson, Chang Dong Yoo

摘要

Automatic Speech Recognition (ASR) systems have attained unprecedented performance with large speech models pre-trained based on self-supervised speech representation learning. However, these pre-trained speech models suffer from representational bias as they tend to better represent those prominent accents (i.e., native (L1) English accent) in the pre-training speech corpus than less represented accents, resulting in a deteriorated performance for nonnative (L2) English accents. Although there have been some approaches to mitigate this issue, all of these methods require updating the pre-trained model weights. In this paper, we propose Information Theoretic Adversarial Prompt Tuning (INTapt), which introduces prompts concatenated to the original input that can re-modulate the attention of the pre-trained model such that the corresponding input resembles a native (L1) English speech without updating the backbone weights. INTapt is trained simultaneously in the following two manners: (1) adversarial training to reduce accent feature dependence between the original input and the prompt-concatenated input and (2) training to minimize CTC loss for improving ASR performance to a prompt-concatenated input. Experimental results show that INTapt improves the performance of L2 English and increases feature similarity between L2 and L1 accents.

问 Lune

几分钟读透这篇 Paper,不用再花一下午。

Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。

已有 1 篇顶会 Paper 引用了它。别等审稿人问起,先弄清它们发现了什么。

可以从这些问题问起

Lune 可以查阅

全文 · 引用它的顶会 Paper · 相关的 Paper

获取答案

免费开始,无需绑卡

脉络

  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale 引用了 Momentum Contrast for Unsupervised Visual Representation Learning
  • Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo 引用了 An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo 引用了 Momentum Contrast for Unsupervised Visual Representation Learning
  • C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion 引用了 SMSMix: Sense-Maintained Sentence Mixup for Word Sense Disambiguation
它引用的引用它的从左到右按年份排列,引用越多,位置越高。

引用它的顶会 Paper1

  1. C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature DispersionHee Suk Yoon, Eunseop Yoon, Joshua Tian Jin Tee, Mark A. Hasegawa-Johnson 等ICLR 2024 · 84 次引用

它引用的 Paper5

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 21,477 次引用
  2. Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCoChaoning Zhang, Kang Zhang, Trung X. Pham, Axi Niu 等CVPR 2022 · 40 次引用
  3. SMSMix: Sense-Maintained Sentence Mixup for Word Sense DisambiguationHee Suk Yoon, Eunseop Yoon, John B. Harvill, Sunjae Yoon 等EMNLP Findings 2022 · 4 次引用
  4. Prefix-Tuning: Optimizing Continuous Prompts for GenerationXiang Lisa Li, Percy LiangACL 2021
  5. Momentum Contrast for Unsupervised Visual Representation LearningKaiming He, Haoqi Fan, Yuxin Wu, Saining Xie 等CVPR 2020

相关 Paper

  1. How Accents Confound: Probing for Accent Information in End-to-End Speech Recognition SystemsArchiki Prasad, Preethi JyothiACL 2020 · 18 次引用
  2. Accented Speech Recognition With Accent-specific CodebooksDarshan Prabhu, Preethi Jyothi, Sriram Ganapathy, Vinit UnniEMNLP 2023 · 5 次引用
  3. ADEPT: A DEbiasing PrompT FrameworkKe Yang, Charles Yu, Yi Ren Fung, Manling Li 等AAAI 2023 · 40 次引用
  4. Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social BiasesYingji Li, Mengnan Du, Xin Wang, Ying WangACL 2023 · 12 次引用
  5. Intuitive Multilingual Audio-Visual Speech Recognition with a Single-Trained ModelJoanna Hong, Se Jin Park, Yong Man RoEMNLP Findings 2023 · 5 次引用
黄昏的海面,两侧是细线勾勒的悬崖