RecSSD: near data processing for solid state drive based recommendation inference
Mark Wilkening, Udit Gupta, Samuel Hsia, Caroline Trippel, Carole-Jean Wu, David Brooks, Gu-Yeon Wei
Abstract
Neural personalized recommendation models are used across a wide variety of datacenter applications including search, social media, and entertainment. State-of-the-art models comprise large embedding tables that have billions of parameters requiring large memory capacities. Unfortunately, large and fast DRAM-based memories levy high infrastructure costs. Conventional SSD-based storage solutions offer an order of magnitude larger capacity, but have worse read latency and bandwidth, degrading inference performance. RecSSD is a near data processing based SSD memory system customized for neural recommendation inference that reduces end-to-end model inference latency by 2× compared to using COTS SSDs across eight industry-representative models.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
31 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- RecShard: statistical feature-based memory optimization for industry-scale neural recommendation cites DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference
- RecShard: statistical feature-based memory optimization for industry-scale neural recommendation cites RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
- SmartSAGE: training large-scale graph neural networks using in-storage processing architectures cites RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
- λ-IO: A Unified IO Stack for Computational Storage cites XRP: In-Kernel Storage Functions with eBPF
Cited by top-tier papers31
- XRP: In-Kernel Storage Functions with eBPFYuhong Zhong, Haoyu Li, Yu Jian Wu, Ioannis Zarkadas et al.OSDI 2022 · 100 citations
- RecShard: statistical feature-based memory optimization for industry-scale neural recommendationGeet Sethi, Bilge Acun, Niket Agarwal, Christos Kozyrakis et al.ASPLOS 2022 · 65 citations
- SmartSAGE: training large-scale graph neural networks using in-storage processing architecturesYunjae Lee, Jinha Chung, Minsoo RhuISCA 2022 · 57 citations
- λ-IO: A Unified IO Stack for Computational StorageZhe Yang, Youyou Lu, Xiaojian Liao, Youmin Chen et al.FAST 2023 · 54 citations
- NVMeVirt: A Versatile Software-defined Virtual NVMe DeviceSang-Hoon Kim, Jaehoon Shim, Euidong Lee, Seong-Yeob Jeong et al.FAST 2023 · 52 citations
Builds on3
- RecNMP: Accelerating Personalized Recommendation with Near-Memory ProcessingLiu Ke, Udit Gupta, Benjamin Youngjae Cho, David Brooks et al.ISCA 2020 · 235 citations
- DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation InferenceUdit Gupta, Samuel Hsia, Vikram Saraph, Xiaodong Wang et al.ISCA 2020 · 149 citations
- NeuMMU: Architectural Support for Efficient Address Translations in Neural Processing UnitsBongjoon Hyun, Youngeun Kwon, Yujeong Choi, John Kim et al.ASPLOS 2020 · 29 citations
Related papers
- RM-SSD: In-Storage Computing for Large-Scale Recommendation InferenceXuan Sun, Hu Wan, Qiao Li, Chia-Lin Yang et al.HPCA 2022 · 33 citations
- TRiM: Enhancing Processor-Memory Interfaces with Scalable Tensor Reduction in MemoryJaehyun Park, Byeongho Kim, Sungmin Yun, Eojin Lee et al.MICRO 2021 · 70 citations
- Enabling Efficient Large Recommendation Model Training with Near CXL Memory ProcessingHaifeng Liu, Long Zheng, Yu Huang, Jingyi Zhou et al.ISCA 2024 · 24 citations
- Training personalized recommendation systems from (GPU) scratch: look forward not backwardsYoungeun Kwon, Minsoo RhuISCA 2022 · 24 citations
- iMARS: an in-memory-computing architecture for recommendation systemsMengyuan Li, Ann Franchesca Laguna, Dayane Reis, Xunzhao Yin et al.DAC 2022 · 14 citations
