Unleashing CPU Potential for Executing GPU Programs Through Compiler/Runtime Optimizations

Although modern CPUs deliver high performance, they are often under-utilized in CPU-GPU heterogeneous systems. Enabling CPUs to execute GPU programs facilitates workload sharing between CPUs and GPUs, which can increase CPU device utilization and overall benefit heterogeneous systems. The flat collapsing transformation represents a state-of-the-art solution for GPU-to-CPU migration and is widely accepted in both academic and commercial projects. However, in this paper, we identify that CPU programs transformed by flat collapsing transformation are not compatible with standard CPU compiler optimizations and runtime environments, which leads to suboptimal performance. Based on the observation, we propose four compiler/runtime optimizations. These optimizations complement flat collapsing transformation and help generate high-performance programs. Our evaluations demonstrate an average performance improvement of 20.84% over the state-of-the-art framework on an x86 CPU and 16.10% on an ARM CPU.