Mantra: Mutation Testing of Hardware Design Code Based on Real Bugs

Mutation testing, a well-suited technology for functional validation, is regrettably poorly studied in hardware. We propose Mantra: the first open-source code-level mutation testing tool based on real hardware bugs. Specifically, Mantra devises time-aware mutation killing mechanism for cost reduction of hardware mutation testing using the parallelism of hardware design code, and then defines and implements 19 hardware mutation operators via large-scale empirical analysis on real bugs. Finally, the evaluation on public datasets from CirFix and OpenCores shows that Mantra achieves promising results with a maximum boost of 83.44%.