dpKernels: Harvesting DPU Compute Resources for Data-path Efficiency in Cloud Data Processing

Data processing units, or DPUs, are equipped with hardware accelerators for compute-intensive data path tasks. Although DPUs' SoC cores are wimpier than the host's, hardware accelerators are typically orders of magnitude faster than CPUs. Harvesting DPU hardware accelerators for database systems could significantly increase throughput and save host CPU cycles. However, due to the heterogeneity of DPUs' hardware configurations and performance characteristics, it is challenging to offer a unified and portable solution for cloud data processing systems to harvest the compute resources on DPUs across generations and vendors. Additionally, due to DPU resource constraints, offloaded compute tasks need to be carefully optimized and scheduled to achieve high efficiency and avoid performance regression. To address these challenges, we introduce two levels of abstraction: dpKernels, which are unified, efficient, and portable primitives that abstract DPU compute resources (i.e., hardware accelerators and SoC cores) for cloud data systems, and dpManager, an onboard management framework that abstracts specific DPU platforms for dpKernels to deliver their promises with optimized, scheduled, and cross-platform executions. The benefits of our proposal have been validated by various workloads, systems, and DPU hardware.