ELMI: Interactive and Intelligent Sign Language Translation of Lyrics for Song Signing
Suhyeon Yoo, Khai N. Truong, Young-Ho Kim
摘要
d/Deaf and hearing song-signers have become prevalent across video-sharing platforms, but translating songs into sign language remains cumbersome and inaccessible. Our formative study revealed the challenges song-signers face, including semantic, syntactic, expressive, and rhythmic considerations in translations. We present ELMI, an accessible song-signing tool that assists in translating lyrics into sign language. ELMI enables users to edit glosses line-by-line, with real-time synced lyric and music video snippets. Users can also chat with a large language model-driven AI to discuss meaning, glossing, emoting, and timing. Through an exploratory study with 13 song-signers, we examined how ELMI facilitates their workflows and how song-signers leverage and receive an LLM-driven chat for translation. Participants successfully adopted ELMI to song-signing, with active discussions throughout. They also reported improved confidence and independence in their translations, finding ELMI encouraging, constructive, and informative. We discuss research and design implications for accessible and culturally sensitive song-signing translation tools.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper1
问问它们各自怎么用它它引用的顶会 Paper6
- Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford, Jong Wook Kim, Tao Xu, Greg Brockman 等ICML 2023 · 被引用 6,966 次
- Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language TranslationRyan Wong, Necati Cihan Camgöz, Richard BowdenICLR 2024 · 被引用 58 次
- "In this online environment, we're limited": : Exploring Inclusive Video Conferencing Design for SignersJazz Rui Xia Ang, Ping Liu, Emma McDonnell, Sarah CoppolaCHI 2022 · 被引用 41 次
- Understanding tensions in music accessibility through song signing for and with d/Deaf and Non-d/Deaf personsSuhyeon Yoo, Georgianna Lin, Hyeon Jeong Byeon, Amy S. Hwang 等CHI 2023 · 被引用 17 次
- Sign Language Transformers: Joint End-to-End Sign Language Recognition and TranslationNecati Cihan Camgöz, Oscar Koller, Simon Hadfield, Richard BowdenCVPR 2020
相关 Paper
- Towards AI-driven Sign Language Generation with Non-manual MarkersHan Zhang, Rotem Shalev-Arkushin, Vasileios Baltatzis, Connor Gillis 等CHI 2025 · 被引用 12 次
- Exploring the Potential of Music Generative AI for Music-Making by Deaf and Hard of Hearing PeopleYoujin Choi, JaeYoung Moon, Jinyoung Yoo, Jin-Hyuk HongCHI 2025 · 被引用 15 次
- Social App Accessibility for Deaf SignersKelly Mack, Danielle Bragg, Meredith Ringel Morris, Maarten W. Bos 等CSCW 2020 · 被引用 52 次
- SCOPE: Sign Language Contextual Processing with Embedding from LLMsYuqi Liu, Wenqian Zhang, Sihan Ren, Chengyu Huang 等AAAI 2025 · 被引用 7 次
- MAVL: A Multilingual Audio-Video Lyrics Dataset for Animated Song TranslationWoohyun Cho, Youngmin Kim, Sunghyun Lee, Youngjae YuEMNLP 2025
