Vision-Guided Action: Enhancing 3D Human Motion Prediction with Gaze-informed Affordance in 3D Scenes

Figure 1. The proposed GAP3DS: Gaze-informed Affordance for human pose Prediction in 3D Scenes. Compared to the state-of-the- art SIF3D [41] utilizing the position of human gaze (yellow cross), GAP3DS is able to identify the intended object (green) and infer its affordance, where both position and affordance information are used to guide the motion prediction. It therefore achieves more accurate predicted poses, and performs significant gains for those poses where human-object interaction is expected (as highlighted in circled areas).