MusicalPT: Augmenting Physical Therapy by Integrating Adaptive Musical Guidance to Enhance Exercise Quality and Patient Experience

Physical Therapy (PT) is crucial for recovery from acute injuries and plays a vital role in promoting functional ability. However, at-home PT is often associated with boredom, lack of engagement, and difficulties in receiving immediate feedback about exercise performance, all of which contribute to poorer PT outcomes. This paper introduces an approach that leverages the engaging, therapeutic power of music to provide intuitive, real-time feedback and adaptive guidance during PT. Developed through a user-centered, research-through-design process, our "MusicalPT" system tracks and sonifies limb movements using computer vision and music generation techniques. We present a low-cost, lightweight, and robust tracking setup that can be used at home with a webcam and a computing device. Our lab-based evaluation shows that this musical guidance helps people perform exercises correctly and remain engaged, while promoting enjoyment and other dimensions of positive user experience. Based on these findings, we discuss broader design directions for interactive musical technologies in the context of health management.