LLMSQLMUTATOR: LLM-Powered Test Case Generation for Database Using Bug Reports

Relational Database Management Systems (DBMSs) are fundamental to modern software infrastructure, yet their inherent complexity and continuous evolution introduce critical defects that compromise data integrity and system reliability. While existing syntax-based and mutation-based automated testing approaches have demonstrated utility in DBMS validation, they suffer from three significant limitations, that are low quality seed SQLs, lack of bug knowledge, and ignorance of fine-grained constraints. To address these limitations, we propose LLM-SQLMutator, an innovative mutation-based automated DBMS testing tool that leverages large language models (LLMs) and bug reports. LLMSQLMUTATOR extracts the SQL statements from the bug report as mutation seeds, and extracts bug patterns and root causes from the bug reports as mutation guidance. Considering the semantic understanding capability of LLMs, we introduce bug-driven SQL mutation which uses LLMs to perform syntax-aware directed mutations on seed SQLs under the guidance of the bug knowledge. Finally, we propose the LLMbased semantic validation that uses database metadata as the basic validation knowledge and dynamically expand validation rules via LLMs, thereby helping LLMSQLMutator achieve comprehensive semantic validation. Extensive evaluations across six popular relational DBMSs demonstrate the advantages of LLMSQLMutator, and it detects 27 confirmed bugs.