ChatBR: Automated assessment and improvement of bug report quality using ChatGPT

Bug reports, containing crucial information such as the Observed Behavior (OB), the Expected Behavior (EB), and the Steps to Reproduce (S2R), can help developers localize and fix bugs efficiently. However, due to the increasing complexity of some bugs and the limited experience of some reporters, many bug reports miss this crucial information. Although machine learning (ML)-based and information retrieval (IR)-based approaches have been proposed to detect and supplement the missing information in bug reports, the performance of these approaches depends heavily on the size and quality of bug report datasets.

In this paper, we present ChatBR, an approach for automated assessment and improvement of bug report quality using ChatGPT. First, we fine-tune a BERT model using manually annotated bug reports to create a sentence-level multi-label classifier to assess the quality of bug reports by detecting the presence of OB, EB, and S2R. Second, we use ChatGPT in a zero-shot setup to generate the missing information (OB, EB, and S2R) to improve the quality of bug reports. Finally, the output of ChatGPT is fed back into the classifier for verification until ChatGPT generates the missing information. Experimental results demonstrate ChatBR's superiority in both detecting and generating missing information in bug reports. For detection, ChatBR surpasses the state-of-the-art method, improving precision by 25.38% to 29.20%. In generating missing information, ChatBR achieves an average semantic similarity of 77.62% between generated and original content across six diverse projects. Furthermore, ChatBR can generate more than 99.9% of