NetSparse: In-Network Acceleration of Distributed Sparse Kernels

Many hardware accelerators have been proposed to accelerate sparse computations. When these accelerators are placed in the nodes of a large cluster, distributed sparse applications become heavily communication-bound. Unfortunately, software solutions to optimize network communication are inefficient.

In this paper, we introduce novel hardware mechanisms to optimize network communication in distributed sparse computations. Our proposal, called NetSparse, consists of four mechanisms. Communication is offloaded to new processing units in the NIC that support efficient remote indexed gather operations, minimizing host-NIC communication. Moreover, these units have the ability to identify and eliminate redundant requests, which minimizes traffic. Further, new hardware modules in the NICs and switches concatenate multiple requests with the same destination node in a single packet, saving traffic and header overheads. Finally, switches are augmented with a hardware cache that stores fetched data from remote racks, making it available to all the nodes in the local rack on demand. Our evaluation on a simulated 128-node cluster with per-node sparse accelerators running sparse workloads reveals that NetSparse improves performance substantially. When the cluster uses traditional software-based communication, the workloads run only 3x faster than on a single-node system; when it is augmented with the NetSparse hardware, the workloads run 38x faster than on the single-node system-attaining more than half of the performance of an ideal system that has no communication overheads.