EvoCrawl: Exploring Web Application Code and State using Evolutionary Search

—As more critical services move onto the web, it has become increasingly important to detect and address vulnerabilities in web applications. These vulnerabilities only occur under specific conditions: when 1) the vulnerable code is executed and 2) the web application is in the required state. If the application is not in the required state, then even if the vulnerable code is executed, the vulnerability may not be triggered. Previous work naively explores the application state by filling every field and triggering every JavaScript event before submitting HTML forms. However, this simplistic approach can fail to satisfy constraints between the web page elements, as well as input format constraints. To address this, we present EvoCrawl, a web crawler that uses evolutionary search to efficiently find different sequences of web interactions. EvoCrawl finds sequences that can successfully submit inputs to web applications and thus explore more code and server-side states than previous approaches. To assess the benefits of EvoCrawl, we evaluate it against three state-of-the-art vulnerability scanners on ten web applications. We find that EvoCrawl achieves better code coverage due to its ability to execute code that can only be executed when the application is in a particular state. On average, EvoCrawl achieves a 59% increase in code coverage and successfully submits HTML forms 5 × more frequently than the next best tool. By integrating IDOR and XSS vulnerability scanners, we used EvoCrawl to find eight zero-day IDOR and XSS vulnerabilities in WordPress, HotCRP, Kanboard, ImpressCMS, and GitLab.