Visar: Projecting Virtual Sound Spots for Acoustic Augmented Reality Using Air Nonlinearity

Augmented reality that integrates virtual content in real-world surroundings has attracted lots of concentration as the growing trend of the metaverse. Acoustic augmented reality (AAR) applications have proliferated due to readily available earphones and speakers. AAR can provide omnidirectional engagement through the all-around sense of spatial information. Most existing AAR approaches offer immersive experiences by playing binaural spatial audios according to head-related transfer functions (HRTF). These involve complex modeling and require the user to wear a headphone. Air nonlinearity that can reproduce audible sounds from ultrasound offers opportunities to achieve device-free and omnidirectional sound source projection in AAR. This paper proposes Visar, a device-free virtual sound spots projection system leveraging air nonlinearity. Visar achieves simultaneous tracking and sound spot generation while suppressing unintended audio leakages caused by grating lobes and nonlinear effects in mixing lobes through optimization. Considering multi-user scenarios, Visar also proposed a multi-spot scheduling scheme to mitigate the mutual interference between the spots. Extensive experiments show the tracking error is 7.83cm and the orientation estimation error is 10.06°, respectively, envisioning the considerable potential of Visar in AAR applications.