Barre Chord: Efficient Virtual Memory Translation for Multi-Chip-Module GPUs

With the advancement of processor packaging technology and the looming end of Moore’s law, multi-chip-module (MCM) GPUs become a promising architecture to continue the performance scaling. However, due to the increasing concurrency, it is challenging to achieve scalable performance. In this study, we show that the limited parallelism in IOMMU is one of the critical bottlenecks and propose Barre Chord to fundamentally reduce the translation loads. By leveraging the unique GPU execution model and page mapping on MCM-GPUs, Barre translates virtual addresses in a unit of coalescing group. Once one page is translated, all the other pages within the same coalescing group can be translated with simple calculations without page table walks. Full Barre (F-Barre) further reduces translations by enabling intra-MCM translation through coalescing information sharing across GPU chiplets and contiguity-aware coalescing group expansion. With the combination of Barre and F-Barre, the Barre Chord outperforms state-of-the-art solutions by an average of 1.36× (2.09× with coalescing group expansion) with negligible area overhead (4.22% of a GPU L2 TLB).