HPIPainting: A Hand-Pen Interaction for VR Painting

Virtual Reality (VR) painting applications allow users to create visual imagery in 3D space. However, existing bare-hand VR painting and sketching systems frequently rely on generic hand gestures, which could lead to significant user misunderstandings. In this paper, we propose HPIPainting, a bare-hand VR painting system that implements a virtual pen interaction mechanism based on the Hand-Pen Interaction(HPI) paradigm. This paradigm leverages microgesture recognition to integrate natural pen-grasping gestures into a precise interaction model, enabling users to sketch in 3D space naturally and immersively. Specifically, we explored the design space for VR painting gestures and filtered out 9 microgestures through subjective evaluations to control various painting functions, such as start painting, edit, brush adjustment, geometric creation, and grid operations. Studies demonstrate that HPIPainting improves the immersion, usability, and expressive freedom of bare-hand VR Painting, it achieves controller-level drawing accuracy with a mean error of 1.21 mm, fast and reliable mode-switching within 222 ms, and significantly higher ratings for ease of use, hand fatigue, and naturalness compared to pinch- and controller-based input.