EMNLP Findings2022
RedApt: An Adaptor for wav2vec 2 EncodingFaster and Smaller Speech Translation without Quality Compromise
Jinming Zhao, Hao Yang, Gholamreza Haffari, Ehsan Shareghi
Abstract
Pre-trained speech Transformers in speech translation (ST) have facilitated state-of-theart (SotA) results; yet, using such encoders is computationally expensive. To improve this, we present a novel Reducer Adaptor block, RedApt, that could be seamlessly integrated within any Transformer-based speech encoding architecture. Integrating the pretrained WAV2VEC 2 speech encoder with RedApt brings 41% speedup, 33% memory reduction with 24% fewer FLOPs at inference. To our positive surprise, our ST model with RedApt outperforms the SotA architecture by an average of 0.68 BLEU score on 8 language pairs from Must-C.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on7
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech RepresentationsAlexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, Michael AuliNeurIPS 2020 · 9,451 citations
- PoWER-BERT: Accelerating BERT Inference via Progressive Word-vector EliminationSaurabh Goyal, Anamitra Roy Choudhury, Saurabh Raje, Venkatesan T. Chakaravarthy et al.ICML 2020 · 260 citations
- PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech RecognitionCheng-I Jeff Lai, Yang Zhang, Alexander H. Liu, Shiyu Chang et al.NeurIPS 2021 · 91 citations
- Adaptive Feature Selection for End-to-End Speech TranslationBiao Zhang, Ivan Titov, Barry Haddow, Rico SennrichEMNLP Findings 2020 · 33 citations
- Multilingual Translation via Grafting Pre-trained Language ModelsZewei Sun, Mingxuan Wang, Lei LiEMNLP Findings 2021 · 18 citations
Related papers
- wav2vec-S: Adapting Pre-trained Speech Models for StreamingBiao Fu, Kai Fan, Minpeng Liao, Yidong Chen et al.ACL Findings 2024 · 6 citations
- Unified Speech-Text Pre-training for Speech Translation and RecognitionYun Tang, Hongyu Gong, Ning Dong, Changhan Wang et al.ACL 2022 · 104 citations
- Multilingual Speech Translation from Efficient Finetuning of Pretrained ModelsXian Li, Changhan Wang, Yun Tang, Chau Tran et al.ACL 2021
- Latent Speech-Text TransformerYen-Ju Lu, Yashesh Gaur, Wei Zhou, Benjamin Muller et al.ICLR 2026 · 7 citations
- Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation EncodersChen Xu, Bojie Hu, Yanyang Li, Yuhao Zhang et al.ACL 2021
