FixationNet: Forecasting Eye Fixations in Task-Oriented Virtual Environments

Fig. 1: Our model's eye fixation prediction performances in different scenes. The green dot represents the ground truth of eye fixation, the red dot denotes the result of our novel model, FixationNet, and the blue dot refers to the state-of-the-art method [21]. In practice, our model exhibits higher accuracy than the state-of-the-art method.

Abstract-Human visual attention in immersive virtual reality (VR) is key for many important applications, such as content design, gaze-contingent rendering, or gaze-based interaction. However, prior works typically focused on free-viewing conditions that have limited relevance for practical applications. We first collect eye tracking data of 27 participants performing a visual search task in four immersive VR environments. Based on this dataset, we provide a comprehensive analysis of the collected data and reveal correlations between users' eye fixations and other factors, i.e. users' historical gaze positions, task-related objects, saliency information of the VR content, and users' head rotation velocities. Based on this analysis, we propose FixationNet -a novel learning-based model to forecast users' eye fixations in the near future in VR. We evaluate the performance of our model for free-viewing and task-oriented settings and show that it outperforms the state of the art by a large margin of 19.8% (from a mean error of 2.93 • to 2.35 • ) in free-viewing and of 15.1% (from 2.05 • to 1.74 • ) in task-oriented situations. As such, our work provides new insights into task-oriented attention in virtual environments and guides future work on this important topic in VR research.