The Matter of Captchas: An Analysis of a Brittle Security Feature on the Modern Web

The web ecosystem is a fast-paced environment. In this dynamic landscape, new security features are offered one after another to enhance the security and robustness of web applications and the operations they handle. This paper focuses on a fragile but still in-use security feature, text-based CAPTCHAs, that had been wildly used by web applications in the past to protect against automated attacks such as credential stuffing and automated account hijacking. The paper first investigates what it takes to develop automated scanners that can solve previously unseen text-based CAPTCHAs. To this end, we evaluated the possibility of developing and integrating a pre-trained CAPTCHA solver in the automated web scanning process without using a significantly large training dataset. We also performed an analysis of the impact of such autonomous scanners on CAPTCHA-enabled websites. Our analysis showed that using solvable text-based CAPTCHAs on login, contact, and comment pages of websites is not uncommon. In particular, we identified more than 3,000 text-based CAPTCHA websites in critical sectors such as finance, government, and health, involving hundreds of thousands of users. We showed that a web scanner with a pretrained solver could solve more than 20% of previously unseen CAPTCHAs in just one single attempt. This result is worrisome considering the substantial potential to autonomously run the operation across thousands of websites on a daily basis with minimal training. Furthermore, the finding suggests that the integration of autonomous scanning with pre-training and local optimization of models can significantly increase adversaries' asymmetric power to launch their attacks cheaper and faster.