Sign-Language Datasets at Scale: A Comprehensive Survey on Resources, Benchmarks, and Annotation Standards
Yiming Ni, Zhi-Qi Cheng, Jiayu Li, Wei Cheng
Abstract
Sign languages are expressive visual languages used by Deaf and Hard-of-Hearing (DHH) communities. Despite substantial progress in sign-language recognition, translation, and production, advances remain constrained by fragmented datasets, inconsistent annotations, and limited linguistic coverage. Existing benchmarks often fail to reflect real-world communication needs, and systematic analyses of these limitations remain limited. In this survey, we present a comprehensive index of sign-language datasets, covering 120 resources across 35 sign languages. We analyze key challenges such as modality imbalance, annotation granularity, and signer bias, and outline considerations for future dataset design. We also introduce a 24-field Sign-Language Datasheet and release a public GitHub repository (https://github.com/Ginqwerty/Open-Sign-Language) to support standardized documentation and reproducible evaluation. Overall, our work provides a unified and practical foundation for developing inclusive, robust, and scalable sign-language technologies in real-world applications.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 7fc6254f-3076-4b3c-b217-2d63d40d5b12Builds on19
- INCLUDE: A Large Scale Dataset for Indian Sign Language RecognitionAdvaith Sridhar, Rohith Gandhi Ganesan, Pratyush Kumar, Mitesh M. KhapraACM MM 2020 · 144 citations
- Gloss-free Sign Language Translation: Improving from Visual-Language PretrainingBenjia Zhou, Zhigang Chen, Albert Clapés, Jun Wan et al.ICCV 2023 · 123 citations
- Fingerspelling Recognition in the Wild With Iterative Visual AttentionBowen Shi, Aurora Martinez Del Rio, Jonathan Keane, Diane Brentari et al.ICCV 2019 · 76 citations
- Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language ProductionBen Saunders, Necati Cihan Camgöz, Richard BowdenCVPR 2022 · 64 citations
- Improving Continuous Sign Language Recognition with Cross-Lingual SignsFangyun Wei, Yutong ChenICCV 2023 · 46 citations
Related papers
- YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel CorpusGarrett Tanzer, Biao ZhangICLR 2025
- Social App Accessibility for Deaf SignersKelly Mack, Danielle Bragg, Meredith Ringel Morris, Maarten W. Bos et al.CSCW 2020 · 52 citations
- OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across LanguagesPrem Selvaraj, Gokul N. C., Pratyush Kumar, Mitesh M. KhapraACL 2022 · 73 citations
- SCOPE: Sign Language Contextual Processing with Embedding from LLMsYuqi Liu, Wenqian Zhang, Sihan Ren, Chengyu Huang et al.AAAI 2025 · 7 citations
- Open-Domain Sign Language Translation Learned from Online VideoBowen Shi, Diane Brentari, Gregory Shakhnarovich, Karen LivescuEMNLP 2022 · 39 citations
