Signals of Aggression: Modelling Multimodal Cues and Perceptual Effects in Virtual Agents

Aggression is a socially complex behaviour that intelligent virtual agents (IVAs) must convincingly convey in applications such as customer service and conflict training. Despite its importance, aggression remains understudied: prior work has focused on basic emotions and unimodal cues, providing little insight into how aggression can be modelled multimodally or systematically scaled by intensity. We present a psychologically grounded model that parametrises language, voice, body movement and facial expressions, across four aggression levels. We evaluated the model in two studies with 38 flight attendants. Experiment 1 tested unimodal cues, showing all modalities except language conveyed aggression gradients. Experiment 2 extended this by combining modalities, demonstrating that coordinated multimodal integration stabilised weaker language cues and produced perceptually robust aggression levels (low, mid, and high) with body and facial cues carrying most weight. Our work contributes the first validated multimodal, multi-level aggression model for IVAs, offering design principles for broader socially expressive agents.