COAXIAL: A CXL-Centric Memory System for Scalable Servers

The memory system is a major performance determinant for server processors. Ever-growing core counts and datasets demand higher memory bandwidth and capacity. DDR—the dominant processor interface to memory—requires a large number of on-chip pins, which is a scarce resource, thus limiting the processor’s memory bandwidth. With limited bandwidth, multiple concurrent memory requests experience significant queuing delays that often overshadow DRAM’s service time and degrade performance. We present CoaXial, a memory system design for throughput-oriented manycore servers that replaces all of the processor’s DDR interfaces with the pin-efficient CXL interface, which offers $4 \times$ higher bandwidth per pin. While such replacement incurs a considerable latency overhead, we demonstrate that, for many workloads, and with careful integration, CXL’s higher bandwidth more than offsets its latency premium. Our evaluation shows that CoaXial improves the performance of manycore throughput-oriented servers by $1.39 \times$ on average and by up to $3 \times$.