Generative Role-Play Communication Training in Virtual Reality for Autistic Individuals: A Study on Job Coach Experiences in Vocational Training Programs

Our study explores the usability and challenges of a Virtual Reality (VR) and Large Language Model (LLM)-based platform designed for soft skills training in vocational programs for autistic individuals, from the perspective of job coaches. The platform features a VR application that integrates an LLM-powered virtual avatar for role-playing scenarios, alongside a web interface system that enables job coaches to develop training scenarios through prompts. We conducted a two-phase longitudinal study with 13 job coaches. Phase 1 involved workshops to introduce the platform and prompt-writing, and assess initial user experiences, while Phase 2 included interviews to gather insights on the system’s usability after 3 months of use. The findings highlight job coaches’ perceptions of the system’s practicality, the difficulties in integrating the technology into routine coaching, and the challenges faced by non-tech-savvy users. Our study contributes to understanding how VR and LLM tools can be effectively utilized in vocational training.