VIREO: Human-in-the-Loop DBMS Fuzzing with Visualization and LLM Support

DBMSs are critical infrastructures that directly affect the security and correctness of many dependent applications. Recent advances in DBMS fuzzing have improved coverage and efficiency, discovering many bugs. However, in practice, many DBMS fuzzers explore only a limited state space, as many functionalities involve complex constraints that require specific sequences of SQL statements and configurations, which automated methods struggle to satisfy. Moreover, many testing approaches lack an understanding of the DBMS testing state and typically run under a single configuration with fixed fuzzing strategies. Consequently, as testing progresses, coverage growth slows, and potential bugs may remain undetected. In this paper, we propose VIREO, a human-in-the-loop DBMS fuzzing framework with visualization and LLM support with two stages. In the preparation stage, VIREO analyzes source code and documentation, leveraging LLMs to construct mappings between each function module and its associated grammar rules and configurations. During the fuzzing stage, VIREO visualizes module-level coverage and bug distributions to reveal potential test boundaries. Test engineers use their domain knowledge to select modules for focused exploration. VIREO then provides the corresponding grammar rules and configuration adjustments, which engineers review to guide the fuzzer, and may also use to construct SQL queries to further enhance testing of the target modules. We have implemented VIREO and evaluated it on 5 welltested DBMSs like PostgreSQL and MySQL. VIREO discovered a total of 25 previously unknown bugs, all of which were confirmed, and 10 have been fixed. Developers provided positive feedback. For example, PostgreSQL developers acknowledged that one identified bug exposed a planner assumption “previously believed unreachable by crafted input.”