An Empirical Study of Fuzz Harness Degradation

Fuzzing is a widely used technique to automatically test software for potential faults. To fuzz software projects efficiently and effectively, software developers must use fuzz harnesses, i.e., small programs that connect the fuzzer to the project's code under test. However, as projects evolve, it is unclear whether fuzz harnesses are maintained in lockstep or left to stagnate, and whether unmaintained fuzz harnesses gradually degrade in terms of code coverage and bug-finding effectiveness.

In this paper, we focus on OSS-Fuzz, the largest continuous fuzzing platform in practice, which provides harnesses for 510 security-critical open-source C/C ++ projects. These harnesses are usually contributed by project maintainers or external developers, yet their ongoing maintenance is not always ensured. Our analysis shows that, overall, harnesses exhibit only a small reduction in coverage and retain surprising longevity in their ability to uncover bugs. At the same time, we also identify cases where harnesses degrade, analyze their root causes and the involved semantics of the code changes, and categorize them systematically. Finally, we extend OSS-Fuzz and Fuzz Introspector, a companion project to investigate fuzzer performance, with new metrics to automatically detect harness degradation, enabling more effective monitoring of fuzzing quality in evolving projects.