EagleRec: Edge-Scale Recommendation System Acceleration with Inter-Stage Parallelism Optimization on GPUs

Recommendation systems suggest items to users by predicting their preferences based on historical data. The industry traditionally handles large-scale recommendation requests by scaling the number of devices without much concern for a single device’s performance. However, there is a trend for recommendation systems to gradually move from a centralized service to an edge device. The edge-scale recommendation systems have distinct features that are different from traditional large-scale deployments, which poses different challenges to the acceleration of the recommendation system. In this paper, we focus on the edge-scale recommendation system and propose an inter-stage parallelism optimization method deployed on a single GPU. Experiments show that our framework could improve recommendation system throughput by 1.89× 2.2× for different datasets on the GPU.