Hitchhiking Hands: Enabling "Virtually Direct" VR Manipulation by Switching Multiple Hand Avatars with Gaze

Dexterous freehand manipulation in virtual reality offers rich interaction but is limited by physical reach. Existing indirect remote manipulation techniques often sacrifice this dexterity. We address this by defining “virtually direct” manipulation, a conceptual framework for techniques that break from a purely direct or indirect model by decoupling the virtual hand from the physical one. Within this framework, we present Hitchhiking Hands (HH), a novel implementation designed to preserve the rich dexterity of direct manipulation at a distance. HH allows users to instantly switch control between multiple pre-defined virtual hands using gaze. We evaluated HH in two user studies. A first study showed our approach, which consistently maintains direct-touch properties, surpasses an established baseline in 6DoF manipulation performance and embodiment. A second qualitative study revealed HH excels in structured spaces but is ill-suited for unstructured global tasks, highlighting a trade-off between flexibility and learnability.