Sassy: SmartNIC-Assisted Notification Delivery for μs-Scale RDMA Workloads

RDMA is the preferred ultra-low-latency transport in modern datacenters. As services decompose into latency-critical microservices, servers maintain hundreds to thousands of connections across tiers. However, RDMA's polling-based RPC notification struggles at high core and connection counts, forcing a tradeoff between idle polling, inter-core synchronization costs, and load imbalance—each limiting throughput for$\mu \mathrm{s}$-scale services. We introduce Sassy, a smartNIC-based notification mechanism that breaks this tradeoff, optimizing the performance and scalability of servicing$\mu \mathrm{s}$-scale RPCs over high connection counts. Sassy eliminates performance overheads stemming from idle polling and inter-core synchronization, while balancing the load of active RDMA connections across available cores. We implement Sassy on an FPGA-enhanced NIC and evaluate it on a range of traffic patterns, as well as a high-performance key-value store. For deployment scenarios with high connection counts, Sassy improves throughput under SLO by up to$2.3 \times$.