Regular Expression Denial of Service Induced by Backreferences

This paper presents the a systematic and theoretical study of denial-of-service vulnerabilities in Regular Expressions with Backreferences (REwB). We introduce the Two-Phase Memory Automaton (2PMFA), an automaton model that precisely captures REwB semantics. Using this model, we derive necessary conditions under which backreferences induce super-linear backtracking runtime, even when sink ambiguity is linear, a regime where existing detectors report no vulnerability. Based on these conditions, we identify three vulnerability patterns and validate them in practice. Using the Snort intrusion detection ruleset, our evaluation identifies 48 previously unknown REwB vulnerabilities with quadratic or worse runtime. We further demonstrate practical exploits against Snort, including slowing rule evaluation by 0.6-1.2 seconds and bypassing alerts by triggering PCRE's matching limit.