It Didn't Sound Good with My Cochlear Implants: Understanding the Challenges of Using Smart Assistants for Deaf and Hard of Hearing Users

How do deaf and hard of hearing (DHH) individuals use smart assistants (SAs)? Does the prominent use of voice interfaces in most SAs pose unique challenges for DHH users? In this work, we aim to answer these questions by conducting 4 in-depth interviews, as well as collecting survey data from 73 DHH individuals. Our findings show that individuals, even with profound deafness, can leverage SAs to accomplish complex daily tasks. However, we also identified a number of common challenges DHH individuals face when interacting with SAs (e.g., high pitch used in the default SA voice interfaces can be incompatible with hearing aids, difficulty using mobile SAs in public places with loud background noise). Based on these insights, we provide a set of suggestions for designing SAs that can better accommodate a wide range of hearing abilities. Specifically, SAs should provide more customization options to allow the user to tailor their SA to meet their hearing needs over time. For example, using a pitch-frequency test feature, much like audiograms conducted by audiologists, could allow users to calibrate their SA's voice to fit within their optimal range. We also see a need to provide more clear and actionable error messages conveyed beyond audio notifications, such as more meaningful light notifications. These recommendations and findings provide the first step forward toward a more inclusive SA by addressing accessibility needs unique to this group.