Co-Designing Multimodal Systems for Accessible Asynchronous Dance Instruction
Ujjaini Das, Shreya Kappala, Meng Chen, Mina Huh, Amy Pavel
Abstract
Videos make exercise instruction widely available, but they rely on visual demonstrations that blind and low vision (BLV) learners cannot see. While audio descriptions (AD) can make videos accessible, describing movements remains challenging as the AD must convey what to do (mechanics, location, orientation) and how to do it (speed, fluidity, timing). Prior work thus used multimodal instruction to support BLV learners with individual simple movements. However, it is unclear how these approaches scale to dance instruction with unique, complex movements and precise timing constraints. To inform accessible asynchronous dance instruction systems, we conducted three co-design workshops (N=28) with BLV dancers, instructors, and experts in sound, haptics, and AD. Participants designed 8 systems revealing common themes: staged learning to dissect routines, crafting vocabularies for movements, and selectively using modalities—narration for movement structure, sound for expression, and haptics for spatial cues. We conclude with design implications to make learning dance accessible.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 49bfb6e4-d228-4c95-bca1-15ed4e167679Builds on23
- MotionGPT: Human Motion as a Foreign LanguageBiao Jiang, Xin Chen, Wen Liu, Jingyi Yu et al.NeurIPS 2023 · 698 citations
- Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion ModelsRongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren et al.ICML 2023 · 469 citations
- Toward Automatic Audio Description Generation for Accessible VideosYujia Wang, Wei Liang, Haikun Huang, Yongqi Zhang et al.CHI 2021 · 86 citations
- What Makes Videos Accessible to Blind and Visually Impaired People?Xingyu Liu, Patrick Carrington, Xiang 'Anthony' Chen, Amy PavelCHI 2021 · 78 citations
- Rescribe: Authoring and Automatically Editing Audio DescriptionsAmy Pavel, Gabriel Reyes, Jeffrey P. BighamUIST 2020 · 72 citations
Related papers
- Sensing Movement: Contemporary Dance Workshops with People who are Blind or have Low Vision and Dance TeachersMadhuka Thisuri De Silva, Jim Smiley, Sarah Goodwin, Leona M. Holloway et al.CHI 2025 · 4 citations
- "It's Kind of Context Dependent": Understanding Blind and Low Vision People's Video Accessibility Preferences Across Viewing ScenariosLucy Jiang, Crescentia Jung, Mahika Phutane, Abigale Stangl et al.CHI 2024 · 23 citations
- Project TapTap: A Longitudinal Study Exploring Non-Verbal Communication through Vibration Signals Between Teachers and Blind or Low Vision Music LearnersLeon Lu, Chase Crispin, Ziyue Piao, Aino Eze-Anyanwu et al.CHI 2025 · 7 citations
- Haptic-Captioning: Using Audio-Haptic Interfaces to Enhance Speaker Indication in Real-Time Captions for Deaf and Hard-of-Hearing ViewersYiwen Wang, Ziming Li, Pratheep Kumar Chelladurai, Wendy Dannels et al.CHI 2023 · 32 citations
- The Design Space of the Auditory Representation of Objects and Their Behaviours in Virtual Reality for Blind PeopleJoão Guerreiro, Yujin Kim, Rodrigo Nogueira, Seung A. Chung et al.IEEE VR 2023 · 32 citations
