Quantifying Social Connection With Verbal and Non-Verbal Behaviors in Virtual Reality Conversations

As virtual reality (VR) continues to evolve as a platform for gathering and collaboration, new forms of communication using voice and avatars are being actively studied. However, the objective and dynamic assessment of social experiences in VR remains a significant challenge, while obtrusive self-report methods prevail. This study aims to identify verbal and nonverbal behavioral indices of perceived social experience in the context of virtual conversations. In our experiment, 52 participants engaged in a ten-minute dyadic conversation in VR and rated the level of social experiences, while turn-taking patterns and behavioral (gaze, pose) data were recorded. The results indicated that rapid response time, longer speech duration, longer gaze duration during turn-taking gaps, and higher nodding frequency during turns predicted the dynamic changes in users’ social experience. By providing objective and unobtrusive measures of social interactions, this study contributes to enhancing the understanding and improvement of social VR experiences.