SeaCache: Efficient and Adaptive Caching for Sparse Accelerators

Sparse tensor computations are highly memory-bound, making on-chip data reuse in SRAM buffers critical to the performance of domain-specific sparse accelerators. On-demand caches are commonly used in recent sparse accelerators, due to the advantage of easy space allocation and the ability to avoid unnecessary data fetches compared to scratchpad-style data buffering. However, existing cache designs suffer from inefficient space utilization due to the difficulty of fitting variable-length sparse data into fixedsize cache blocks. The theoretically optimal replacement policies used by them also have substantial implementation cost, requiring significant on-chip space to manage the metadata. To address these issues, we propose SeaCache to enable efficient and adaptive caching for sparse accelerators. SeaCache includes three key techniques. First, it incorporates fiber packing and splitting to efficiently map variable-length data into fixed-size cache blocks with high space utilization. Second, it proposes a practical replacement policy that performs similarly to the optimal one but has a much cheaper implementation. Third, it shares the cache space between the actual data and the replacement policy metadata, with a two-phase adaptive mechanism to decide the best partition ratio. Overall, Sea-Cache outperforms state-of-the-art sparse cache designs by 2.8× on average, demonstrating the effectiveness of its novel optimizations.