Test-case reduction and deduplication almost for free with transformation-based compiler testing

Recent transformation-based approaches to compiler testing look for mismatches between the results of pairs of equivalent programs, where one program is derived from the other by randomly applying semantics-preserving transformations. We present a formulation of transformation-based compiler testing that provides effective test-case reduction almost for free: if transformations are designed to be as small and independent as possible, standard delta debugging can be used to shrink a bug-inducing transformation sequence to a smaller subsequence that still triggers the bug. The bug can then be reported as a delta between an original and minimallytransformed program. Minimized transformation sequences can also be used to heuristically deduplicate a set of buginducing tests, recommending manual investigation of those that involve disparate types of transformations and thus may have different root causes. We demonstrate the effectiveness of our approach via a new tool, spirv-fuzz, the first compilertesting tool for the SPIR-V intermediate representation that underpins the Vulkan GPU programming model.

• Software and its engineering → Compilers; Software testing and debugging.