NetRT: Enhancing RDMA with Retransmission Offloading in Data Center Networks

RDMA over Converged Ethernet (RoCE) enables the deployment of RDMA in Ethernet-based data centers and is becoming a mainstream solution. Due to the limited processing capability of RDMA NICs (RNICs), Priority-based Flow Control (PFC) is enabled to prevent decreased transmission efficiency caused by packet loss and retransmission. However, PFC brings performance impairments and cannot fully ensure reliability in the presence of various packet loss and out-of-order delivery cases in large-scale data centers. To achieve high resilience against lossy conditions and disorders while considering feasibility, this paper proposes NetRT, an innovative retransmission offloading solution. NetRT deploys selective retransmission between top-of-rack switches instead of RNICs, leveraging modern switches' hardware resources and programmable features for efficient and deployable in-network recovery. We develop key mechanisms to manage in-network retransmissions, including a ternary state machine and congestion avoidance, aiming for efficiency and transparency. Evaluations show that NetRT can handle diverse packet loss and out-of-order cases with acceptable overhead, reducing end - to-end retransmissions and decreasing flow completion times by up to 75 %.