NestFuzz: Enhancing Fuzzing with Comprehensive Understanding of Input Processing Logic

Fuzzing is one of the most popular and practical techniques for security analysis. In this work, we aim to address the critical problem of high-quality input generation with a novel input-aware fuzzing approach called NestFuzz. NestFuzz can universally and automatically model input format specifications and generate valid input.

The key observation behind NestFuzz is that the code semantics of the target program always highly imply the required input formats. Hence, NestFuzz applies fine-grained program analysis to understand the input processing logic, especially the dependencies across different input fields and substructures. To this end, we design a novel data structure, namely Input Processing Tree, and a new cascading dependency-aware mutation strategy to drive the fuzzing.

Our evaluation of 20 intensively-tested popular programs shows that NestFuzz is effective and practical. In comparison with the state-of-the-art fuzzers (AFL, AFLFast, AFL++, MOpt, AFLSmart, WEIZZ, ProFuzzer, and TIFF), NestFuzz achieves outperformance in terms of both code coverage and security vulnerability detection. NestFuzz finds 46 vulnerabilities that are both unique and serious. Until the moment this paper is written, 39 have been confirmed and 37 have been assigned with CVE-ids.