Shepard Steps: Exploiting an Auditory Illusion for Footstep Sounds in Uneven Virtual Reality Walking

A fundamental principle of virtual reality is the crafting of illusion which immerses and guides users. In contrast to the ample work on visual illusions and VR, auditory illusions remain relatively understudied. We are particularly interested in how VR can leverage auditory stimuli to manipulate user movement and perception of movement. Here we conduct trials to study how the sounds of footsteps can be manipulated to give the impression of upwards or downwards incline during walking, and how this compares to visual stimuli. We do so by crafting Shepard tone imbued footstep sounds which exploit the cross-modal correspondence between auditory pitch and elevation. We found that while our chosen visual stimuli were considerably more powerful, our footstep sounds still shifted the perception of movement direction, including when they contradicted the direction of visual stimuli. This study opens new research possibilities on how other auditory illusions can influence movement. The Shepard footsteps themselves have direct application in environments where the user must explore the vertical dimension of a 3D space.