Towards Consumer-Grade Cybersickness Prediction: Multi-Model Alignment for Real-Time Vision-Only Inference
Yitong Zhu, Zhuowen Liang, Yiming Wu, Tangyao Li, Yuyang Wang
摘要
Cybersickness remains a major obstacle to the widespread adoption of immersive virtual reality (VR), particularly in consumer-grade environments. While prior methods rely on invasive signals such as electroencephalography (EEG) for high predictive accuracy, these approaches require specialized hardware and are impractical for real-world applications. In this work, we propose a scalable, deployable framework for personalized cybersickness prediction leveraging only non-invasive signals readily available from commercial VR headsets, including head motion, eye tracking, and physiological responses. Our model employs a modality-specific graph neural network enhanced with a Difference Attention Module to extract temporal-spatial embeddings capturing dynamic changes across modalities. A cross-modal alignment module jointly trains the video encoder to learn personalized traits by aligning video features with sensor-derived representations. Consequently, the model accurately predicts individual cybersickness using only video input during inference. Experimental results show our model achieves 88.4% accuracy, closely matching EEG-based approaches (89.16%), while reducing deployment complexity. With an average inference latency of 90ms, our framework supports real-time applications, ideal for integration into consumer-grade VR platforms without compromising personalization or performance. The code will be relesed at https://github.com/U235-Aurora/PTGNN.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper4
- A Deep Cybersickness Predictor Based on Brain Signal Analysis for Virtual Reality ContentsJinwoo Kim, Woojae Kim, Heeseok Oh, Seongmin Lee 等ICCV 2019 · 被引用 90 次
- PRECYSE: Predicting Cybersickness using Transformer for Multimodal Time-Series Sensor DataDayoung Jeong, Kyungsik HanUbiComp 2024 · 被引用 35 次
- LiteVR: Interpretable and Lightweight Cybersickness Detection using Explainable AIRipan Kumar Kundu, Rifatul Islam, John Quarles, Khaza Anuarul HoqueIEEE VR 2023 · 被引用 34 次
- Hypergraph Multi-modal Large Language Model: Exploiting EEG and Eye-tracking Modalities to Evaluate Heterogeneous Responses for Video UnderstandingMinghui Wu, Chenxu Zhao, Anyang Su, Donglin Di 等ACM MM 2024 · 被引用 8 次
相关 Paper
- Investigating Personalization Techniques for Improved Cybersickness Prediction in Virtual Reality EnvironmentsUmama Tasnim, Rifatul Islam, Kevin Desai, John QuarlesIEEE VR 2024 · 被引用 29 次
- Beyond Subjectivity: Continuous Cybersickness Detection Using EEG-based Multitaper Spectrum EstimationBerken Utku Demirel, Adnan Harun Dogan, Juliete Rossie, Max Möbus 等IEEE VR 2025 · 被引用 5 次
- Predicting Cybersickness Trend and Extent Based on FMS Labeled DatasetJun Ryu, Gerard J. KimIEEE VR 2026 · 被引用 1 次
- A Statistical Abstraction Framework for Integrating Heterogeneous VR Datasets in Ordinal Cybersickness PredictionJyotirmay Nag Setu, John QuarlesIEEE VR 2026
- Omnidirectional Galvanic Vestibular Stimulation in Virtual RealityColin Groth, Jan-Philipp Tauscher, Nikkel Heesen, Max Hattenbach 等IEEE VR 2022 · 被引用 51 次
