Instruction-Aware Cooperative TLB and Cache Replacement Policies

Modern server and data center applications are characterized not only by large datasets, but also by large instruction footprints that incur frequent cache and Translation Lookaside Buffer (TLB) misses due to instruction accesses. Instruction TLB misses are particularly problematic as they cause pipeline stalls that significantly harm performance.

This paper proposes cooperative last-level TLB (STLB) and L2 cache (L2C) replacement policies targeting workloads with large instruction footprints. We propose Instruction Translation Prioritization (iTP), an STLB replacement policy that maximizes the number of instruction hits in the STLB at the expense of increasing data page walks. To compensate for the increase in data page walks, we propose extended Page Table Prioritization (xPTP), a new L2C replacement policy that amplifies the benefits of iTP by effectively reducing L2C misses due to data page walks. Our proposal, iTP+xPTP, combines iTP at STLB and xPTP at L2C. iTP+xPTP employs an adaptive mechanism that switches between xPTP and LRU policies at L2C based on the pressure placed on the virtual memory subsystem. Our proposal, iTP+xPTP, improves single-core geometric mean performance by 18.9% over a baseline that uses the LRU replacement policy at both STLB and L2C across a set of contemporary server workloads. Under SMT co-location, the corresponding performance uplift is 11.4%. Finally, we show that iTP+xPTP outperforms the state-of-the-art STLB and cache replacement policies.