Towards Building Large Scale Datasets and State-of-the-Art Automatic Speech Translation Systems for 14 Indian Languages
Ashwin Sankar, Sparsh Jain, Nikhil Narasimhan, Devilal Choudhary, Dhairya Suman, Mohammed Safi Ur Rahman Khan, Anoop Kunchukuttan, Mitesh M. Khapra, Raj Dabre
Abstract
Speech translation for Indian languages remains a challenging task due to the scarcity of large-scale, publicly available datasets that capture the linguistic diversity and domain coverage essential for real-world applications. Existing datasets cover a fraction of Indian languages and lack the breadth needed to train robust models that generalize beyond curated benchmarks. To bridge this gap, we introduce BhasaAnuvaad, the largest speech translation dataset for Indian languages, spanning over 44 thousand hours of audio and 17 million aligned text segments across 14 Indian languages and English. Our dataset is built through a threefold methodology: (a) aggregating high-quality existing sources, (b) large-scale web crawling to ensure linguistic and domain diversity, and (c) creating synthetic data to model real-world speech disfluencies. Leveraging BhasaAnuvaad, we train IndicSeamless, a state-of-the-art speech translation model for Indian languages that performs better than existing models. Our experiments demonstrate improvements in the translation quality, setting a new standard for Indian language speech translation. We will release all the code, data and model weights in the open-source, with permissive licenses to promote accessibility and collaboration.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext a11a93cd-6f63-4b26-a88f-662fc11c60a9Cited by top-tier papers2
- UPDESH: Synthesizing Grounded Instruction Tuning Data for 13 Indic LanguagesPranjal A. Chitale, Varun Gumma, Sanchit Ahuja, Prashant Kodali et al.ACL 2026 · 1 citation
- Scalable Multilingual Multimodal Machine Translation with Speech-Text FusionYexing Du, Youcheng Pan, Zekun Wang, Zheng Chu et al.ICLR 2026
Builds on6
- Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford, Jong Wook Kim, Tao Xu, Greg Brockman et al.ICML 2023 · 6,966 citations
- Direct Speech-to-Speech Translation With Discrete UnitsAnn Lee, Peng-Jen Chen, Changhan Wang, Jiatao Gu et al.ACL 2022 · 235 citations
- SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech TranslationsPaul-Ambroise Duquenne, Hongyu Gong, Ning Dong, Jingfei Du et al.ACL 2023 · 18 citations
- IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation Metrics for Indian LanguagesAnanya B. Sai, Tanay Dixit, Vignesh Nagarajan, Anoop Kunchukuttan et al.ACL 2023 · 8 citations
- Speech Translation and the End-to-End Promise: Taking Stock of Where We AreMatthias Sperber, Matthias PaulikACL 2020 · 7 citations
Related papers
- From Tens of Hours to Tens of Thousands: Scaling Back-Translation for Speech RecognitionTianduo Wang, Lu Xu, Wei Lu, Shanbo ChengEMNLP 2025 · 1 citation
- Towards Building ASR Systems for the Next Billion UsersTahir Javed, Sumanth Doddapaneni, Abhigyan Raman, Kaushal Santosh Bhogale et al.AAAI 2022 · 86 citations
- IndicSynth: A Large-Scale Multilingual Synthetic Speech Dataset for Low-Resource Indian LanguagesDivya V. Sharma, Vijval Ekbote, Anubha GuptaACL 2025 · 6 citations
- IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian LanguagesTahir Javed, Kaushal Santosh Bhogale, Abhigyan Raman, Pratyush Kumar et al.AAAI 2023 · 47 citations
- BhashaSutra: A Task-Centric Unified Survey of Indian NLP Datasets, Corpora, and ResourcesRaghvendra Kumar, Devankar Raj, Sriparna SahaACL 2026
