Cognitive Load Inference Using Physiological Markers in Virtual Reality

Virtual reality (VR) has become an increasingly popular way for learning and training. The assessment of the amount of mental effort, or cognitive load required to perform a task, is essential to create adaptive VR experiences. In this work, we conducted a large-scale study (N=738) to collect behavioral and physiological measures under different cognitive load conditions in a VR environment, and developed a novel machine learning solution to predict cognitive load in real time. Our model predicts cognitive load as a continuous value in the range from 0 to 1, where 0 and 1 correspond to the lowest and highest reported cognitive loads across all participants. On top of the point estimation of cognitive load, our model quantifies prediction uncertainty using a prediction interval. We propose a novel dual-branch attention model to accurately predict the cognitive load. We achieve a MAE (Mean Absolute Error) of 0.11. The result indicates that, with a combination of behavioral and physiological indicators, we can reliably predict cognitive load in real-time, without calibration. To support further research, we are releasing a test dataset comprising data from 100 participants for use by researchers and developers interested in machine learning, virtual reality, learning & memory, cognition, or psychophysiology. This dataset includes recordings from multiple sensors (including pupillometry, eye-tracking, and pulse plethysmography), self-reported cognitive effort, behavioral task performance, and demographic information on the sample.