pPatch: Automated Vulnerability Unpatching

Unpatching, the process of reverting security patches to reintroduce historical vulnerabilities into newer software versions, is valuable for creating realistic benchmarks to evaluate security analysis tools. However, this process is challenging due to code evolution, leading to context conflicts, compilation errors, or untriggerable issues. In fact, 61.25% of Linux kernel security patches we examined cannot be trivially reverted to recent versions. To address this, we propose pPatch, an automated framework designed to systematically unpatch security vulnerabilities and generate vulnerability benchmark. pPatch overcomes the limitations of naive reversion by employing a novel approach that progressively consults conflicting commits to identify and integrate necessary code changes, aiming for minimal modifications to preserve program semantics while successfully re-exposing the original vulnerability and minimizing unintended side effects. Then pPatch unpatches 614 historic kernel vulnerabilities from Linux kernel v6.6 and v6.12, resulting in 371 and 353 successfully unpatched vulnerabilities with manual analysis. Based on the validation with Proof-of-Concept (PoC) and automated fuzzing, we constructed KVulnBench with 187 verified vulnerabilities, the first automatically generated and verified high-quality vulnerability dataset specifically for the Linux kernel. Using KVulnBench we evaluated the performance of state-of-the-art kernel security tools (e.g., syzkaller), demonstrating that KVulnBench provides a valuable resource for realistically assessing kernel security tools.