Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels

Image GT Supervised (Gaze360) image inference ST-WSGE (Gaze360+GF) video inference ST-WSGE (Gaze360+GF) image inference Figure 1. Significance of ST-WSGE. Our self-training based weakly-supervised framework for robust 3D gaze estimation in real-world conditions (e.g., varying appearance, extreme poses, resolution, and occlusion). All predictions used our image and video agnostic Gaze Transformer (GaT) model. Top row: importance of the training diversity using ST-WSGE and GazeFollow (GF) for generalization compared to standard supervised methods. Bottom row: influence of temporal context between image and video inference. Circles in images represent unit disks where 3D gaze vectors are projected onto the image plane (x, y in yellow) and a top-down view (x, z in blue). Images from VideoAttentionTarget, GFIE, and MPIIFaceGaze datasets.