Tuning Immersion and Performance with Adaptive Generative Music in VR

cheerful and playful orchestral music with soft drums and light melodies cinematic background music with steady rhythm and a sense of challenge intense cinematic buildup with fast percussion and escalating strings fast-paced, dramatic orchestral music with rapid strings, intense percussion and a sense of urgency ADAPTIVE MUSIC CONDITION FIXED MUSIC CONDITION cheerful and playful orchestral music with soft drums and light, melodies 20 15 10 5 Num of arrow left Adaptive prompt Fixed prompt Fig. 1: Diagrammtic overview of an experiment contrasting adaptive and fixed music conditions in a VR archery task. In the adaptive condition, prompts in Google's MusicFXDJ evolved dynamically with task progression (e.g., increasing intensity as arrows decreased), while in the fixed condition the same cheerful orchestral track played throughout. This design allowed us to isolate the experiential and performance effects of adaptive versus fixed background music.

Abstract-Music in virtual environments has long been treated as a temporal evolving element, enhancing atmosphere and game pace but rarely considered as a performance adaptive element. Recent advances in artificial intelligence (AI) and procedural audio make it possible to generate music that adapts in real time to player actions and system state. Yet, despite its potential, the behavioural impact of such adaptive generative soundtracks in head-mounted display-based virtual reality (VR) remains largely unexplored.

To address this gap, we introduce a VR archery system that integrates Google MusicFXDJ with Ubiq-Genie to deliver continuous AI-generated adaptive music driven by gameplay events. In a within-subjects experiment (N = 22), participants completed trials with either a stylistically-matched fixed soundtrack or an adaptive soundtrack that escalated tension across four phases as arrows depleted. Measures combined self-reported ratings of presence, focus, stress, and emotional impact, with performance metrics of accuracy and aiming time. Results reveal that adaptive generative music not only heightens immersion and emotional salience but also modulates motor precision in an arousal-dependent inverted-U pattern: moderate musical tension improved accuracy and speed, whereas excessive tension impaired them. These findings establish AI-generated music as a powerful behavioural feedback modality in VR, opening pathways for training, rehabilitation, and next-generation immersive entertainment.