Few-Shot Adaptive Gaze Estimation

Inter-personal anatomical differences limit the accuracy of person-independent gaze estimation networks. Yet there is a need to lower gaze errors further to enable applications requiring higher quality. Further gains can be achieved by personalizing gaze networks, ideally with few calibration samples. However, over-parameterized neural networks are not amenable to learning from few examples as they can quickly over-fit. We embrace these challenges and propose a novel framework for Few-shot Adaptive GaZE Estimation (FAZE) for learning person-specific gaze networks with very few (≤ 9) calibration samples. FAZE learns a rotationaware latent representation of gaze via a disentangling encoder-decoder architecture along with a highly adaptable gaze estimator trained using meta-learning. It is capable of adapting to any new person to yield significant performance gains with as few as 3 samples, yielding state-of-theart performance of 3.18 • on GazeCapture, a 19% improvement over prior art. We open-source our code at https: //github.com/NVlabs/few_shot_gaze 1 . * The first two authors contributed equally. 1 This includes a real-time demo which takes < 10 seconds to record 9 calibration points for a new user and ∼ 1 minute to train a personalized network on a laptop with an NVIDIA GTX GeForce 1060 GPU.