ACL Findings2023
INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech Recognition
Eunseop Yoon, Hee Suk Yoon, John B. Harvill, Mark Hasegawa-Johnson, Chang Dong Yoo
摘要
Automatic Speech Recognition (ASR) systems have attained unprecedented performance with large speech models pre-trained based on self-supervised speech representation learning. However, these pre-trained speech models suffer from representational bias as they tend to better represent those prominent accents (i.e., native (L1) English accent) in the pre-training speech corpus than less represented accents, resulting in a deteriorated performance for nonnative (L2) English accents. Although there have been some approaches to mitigate this issue, all of these methods require updating the pre-trained model weights. In this paper, we propose Information Theoretic Adversarial Prompt Tuning (INTapt), which introduces prompts concatenated to the original input that can re-modulate the attention of the pre-trained model such that the corresponding input resembles a native (L1) English speech without updating the backbone weights. INTapt is trained simultaneously in the following two manners: (1) adversarial training to reduce accent feature dependence between the original input and the prompt-concatenated input and (2) training to minimize CTC loss for improving ASR performance to a prompt-concatenated input. Experimental results show that INTapt improves the performance of L2 English and increases feature similarity between L2 and L1 accents.
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
已有 1 篇顶会 Paper 引用了它。别等审稿人问起,先弄清它们发现了什么。
脉络
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale 引用了 Momentum Contrast for Unsupervised Visual Representation Learning
- Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo 引用了 An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo 引用了 Momentum Contrast for Unsupervised Visual Representation Learning
- C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion 引用了 SMSMix: Sense-Maintained Sentence Mixup for Word Sense Disambiguation
引用它的顶会 Paper1
它引用的 Paper5
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 21,477 次引用
- Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCoChaoning Zhang, Kang Zhang, Trung X. Pham, Axi Niu 等CVPR 2022 · 40 次引用
- SMSMix: Sense-Maintained Sentence Mixup for Word Sense DisambiguationHee Suk Yoon, Eunseop Yoon, John B. Harvill, Sunjae Yoon 等EMNLP Findings 2022 · 4 次引用
- Prefix-Tuning: Optimizing Continuous Prompts for GenerationXiang Lisa Li, Percy LiangACL 2021
- Momentum Contrast for Unsupervised Visual Representation LearningKaiming He, Haoqi Fan, Yuxin Wu, Saining Xie 等CVPR 2020
相关 Paper
- How Accents Confound: Probing for Accent Information in End-to-End Speech Recognition SystemsArchiki Prasad, Preethi JyothiACL 2020 · 18 次引用
- Accented Speech Recognition With Accent-specific CodebooksDarshan Prabhu, Preethi Jyothi, Sriram Ganapathy, Vinit UnniEMNLP 2023 · 5 次引用
- ADEPT: A DEbiasing PrompT FrameworkKe Yang, Charles Yu, Yi Ren Fung, Manling Li 等AAAI 2023 · 40 次引用
- Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social BiasesYingji Li, Mengnan Du, Xin Wang, Ying WangACL 2023 · 12 次引用
- Intuitive Multilingual Audio-Visual Speech Recognition with a Single-Trained ModelJoanna Hong, Se Jin Park, Yong Man RoEMNLP Findings 2023 · 5 次引用
