Effects of Language Familiarity in Simulated Natural Dialogue with a Virtual Crowd of Digital Humans on Emotion Contagion in Virtual Reality

This investigation compared the emotional impact caused by a crowd of affective virtual humans (VHs) that communicated in the users' native or foreign language. We evaluated the users' affective reactions to a crowd of VHs that exhibited distinct emotional expressions. A total of four emotions were presented, which were Positive, Negative, Neutral, and Mixed. The VHs performed verbal and non-behaviors accordingly. Under the Mixed condition, the VHs were divided into three groups equally and each group was uniquely assigned one of the three emotions (i.e., positive, negative, and neutral). Users collected ten items from a virtual reality market. To complete the tasks, they interacted using natural speech with the emotional VHs. Three language conditions were investigated: one condition in USA and another two conditions in Taiwan. The group of participants in USA interacted with the VHs in English; and the two groups of participants in Taiwan interacted with the VHs using a foreign (English) language and a native (Mandarin) language respectively. We discovered that the medium of communication or language familiarity had a strong influence on participants' emotional reactions. When participants interacted in a foreign language with VHs with a positive emotional disposition, we found their positive emotional reactions were subdued and negative reactions were elevated. However, this was not the case when participants interacted with VHs in their native language, as their emotional reactions were contingent on the emotional disposition of the VHs.