GX-Plug: a Middleware for Plugging Accelerators to Distributed Graph Processing

Recently, research communities highlight the necessity of formulating a scalability continuum for large-scale graph processing, which gains the scale-out benefits from distributed graph systems, and the scale-up benefits from high-performance accelerators. To this end, we propose a middleware, called the GX-plug, for the ease of integrating the merits of both. As a middleware, the GX-plug is versatile in supporting different runtime environments, computation models, and programming models. More, for improving the middleware performance, we study a series of techniques, including pipeline shuffle, synchronization caching and skipping, and workload balancing, for intra-, inter-, and beyond-iteration optimizations, respectively. Experiments show that our middleware efficiently plugs accelerators to representative distributed graph systems, e.g., GraphX and Powergraph, with up-to 20x acceleration ratio.

Index Terms-Distributed graph systems, Middleware, accelerators 1 BSP (Bulk Synchronous Parallel) is a parallel model that performs computation in iterative steps, including three steps of computation, communication, and synchronization. BSP model [11] has been the most fundamental and popular execution approach on distributed graph systems. GAS (Gather-Apply-Scatter) model [3] is another basic and widely adopted model for distributed graph processing, based on BSP [12].

2 It includes vertex-and edge-centric models [13].