The Impact of Visual and Haptic Feedback on Keyboard Typing in Immersive Virtual Environments

Typing with a keyboard is a common task in content production in the workplace. For simulation purposes in VR environments, it is important for users to perform this task accurately, with minimal performance loss, and without distraction. One common approach is using mid-air typing on virtual keyboards. However, this method presents challenges, particularly due to the lack of haptic feedback and spatial awareness. Various solutions have been suggested in the literature to address these challenges, but several design factors that influence performance, behavior, and user experience still need to be explored. This paper investigates the effects of two types of visual feedback (hover and keypress) and three passive haptic feedback conditions (physical keyboard, physical surface, and a mid-air virtual keyboard with no haptic feedback) and the possible interactions between these factors on typing using the two index fingers in VR. Results show that keypress visual feedback enhanced typing speed and reduced workload, while hover feedback lowered error rates but negatively impacted typing speed. Additionally, using a physical keyboard to provide passive haptic feedback increased the error rate. This increase in the error rate could be attributed to inaccuracies in finger and keyboard tracking, which may have caused a misalignment between the physical and virtual environments. Regarding eye gaze behavior, participants spent more time looking at the keyboard with the keypress visual feedback and when no haptic feedback was provided. Finally, participants rated the physical keyboard as the least usable option.