Utilizing Generative AI for VR Exploration Testing: A Case Study

As the virtual reality (VR) industry expands, the need for automated GUI testing for applications is growing rapidly. With its long-term memory and ability to process mixed data, including images and text, Generative AI (GenAI) shows the potential to understand complex user interfaces. In this paper, we conduct a case study to investigate the potential of using GenAI for field of view (FOV) analysis in VR exploration testing. Specifically, we examine how the model can assist in test entity selection and test action suggestions. Our experiments demonstrate that while GPT-4o achieves a 63% accuracy rate in object identification within an arbitrary FOV, it struggles with object organization and localization. We also identify critical contexts that can improve the accuracy of suggested actions across multiple FOVs. Finally, we discuss the limitations found during the experiment and offer insights into future research directions.