Watch and Crack: Password Inference from Smart-Glasses Video

Typing passwords on smartphones in public places exposes users to video-based side-channel attacks, in which an adversary records the typing session and reconstructs the entered password by analyzing finger movements. Prior video-based keystroke inference attacks have targeted free-form text on tablets, numeric PINs, pattern locks, and passwords under unrealistically simplified conditions.

In this paper we present the first general video-based keystroke inference attack pipeline that recovers rule-based alphanumeric passwords from smartphone QWERTY keyboards, covering all four keyboard layouts, and requiring no assumptions about the victim or their device. Our pipeline uses the built-in camera of the popular Meta Ray-Ban smart glasses to record a victim typing a password on a smartphone in a public setting. It tracks the device and typing fingertips at sub-pixel resolution, predicts keystrokes via a selfsupervised ensemble of neural networks, dynamically estimates the on-screen keyboard layout, and computes per-key probability distributions. These video-derived probabilities are then combined with prior password-typing distributions based on 2.19 billion leaked passwords to estimate the rank, and therefore the crack time, of the observed password.

We evaluate our pipeline via a user study, against passwords that adhere to a typical enterprise password policy, including both human-chosen and password-manager-chosen random passwords. Users typed naturally, some using two thumbs and some using the index finger. With our video side channel, human-chosen passwords of up to 16 characters can be cracked in hours to days, and password-manager-chosen random passwords of up to 13 characters can be cracked in under an hour on a modern GPU, reducing password entropy by up to 60 bits relative to the prior baseline. For human-chosen passwords, combining video evidence with prior statistics yields an additional statistically significant reduction of approximately 20 bits compared to either source alone. The attack succeeds at distances up to 1.8 m, across diverse attacker-victim postures (seated or standing) and viewing angles (from frontal to side-profile), and generalizes across three popular smartphones (Google Pixel 10, iPhone 16, and Samsung Galaxy A53).