CareAssist: Supporting Offline Caregiver Coaching in Daily Autism Care through Multimodal Sensing and LLMs

Caregiver-mediated early interventions have been shown to benefit children with autism spectrum disorder (ASD). Yet caregivers, who often balance parent and interventionist roles, may struggle to recognize and interpret intervention-worthy moments during dynamic everyday interactions. Relationship-based intervention programs often leverage clinician-led retrospective video review to help caregivers reflect on such moments, but this workflow is difficult to scale because it is time- and expertise-intensive. Meanwhile, existing digital autism tools largely emphasize child-focused skill rehearsal or monitoring, offering limited support for caregiver-centered reflection. We investigate how multimodal sensing and large language models (LLMs) can support the offline video-review coaching workflow. Co-designed with ASD experts, CareAssist fuses synchronized video and photoplethysmography (PPG) signals to detect intervention-worthy moments involving emotional dysregulation or low social engagement and generates grounded moment-level draft guidance and post-session summaries. Through a field study with 56 families, we evaluate moment detection against expert labels and compare three prompting strategies using expert-defined rubrics; we additionally report a pilot caregiver case study as illustrative caregiver-perspective evidence. Quantitative analysis and qualitative feedback show that multimodal detection outperforms baseline models, while structured prompting improves selected expert-rated dimensions and reveals trade-offs among clarity, contextual fit, and ethical suitability. These findings demonstrate the potential of multimodal sensing and LLMs to support scalable offline caregiver coaching beyond clinical environments while identifying safeguards and design requirements for future deployment.