Starrypia: An AR Gamified Music Adjuvant Treatment Application for Children with Autism Based on Combined Therapy

In this paper, we present Starrypia, a lightweight gamified music adjuvant treatment application to improve the symptoms of mild autistic children, eliminating the geographical and time constraints faced by traditional treatment. Adopting ABA (Applied Behavior Analysis) behavioral theory as the principle, Starrypia follows the stimulus-response-reinforcement-pause process and incorporates music therapy and sensory integration. Based on AR, Starrypia provides multi-sensory intervention through music generated by BiLSTM deep model, 3D visual scenes, touch interaction to keep children focused and calm. We conducted a controlled experiment on 20 children to test Starrypia’s effectiveness and attraction. Children’s pre-test and post-test scores on two autism rating scales and performance during the test were applied to measure their abilities and engagement. Experimental results indicated that children showed great interest in Starrypia and presented evident symptom remission and advance in overall abilities after 4 weeks of use. In conclusion, Starrypia is practicable in both therapeutic effect and user experience, and conspicuously instrumental in promoting sensory ability.