NN4SysBench: Characterizing Neural Network Verification for Computer Systems

We present NN4SysBench, a benchmark suite for neural network verification, comprised of benchmarks from neural networks for systems (or NN4Sys). NN4Sys is booming: there are hundreds of proposals of using neural networks in computer systems-databases, OSes, and networked systems-that are safety critical. We observe that NN4Sys has some unique characteristics that today's neural network verification tools overlooked. This benchmark aims at bridging the gap between NN4Sys and NN-verification by tailoring impactful NN4Sys instances to benchmarks that today's NN-verification tools can work on.