Shard Manager: A Generic Shard Management Framework for Geo-distributed Applications
Sangmin Lee, Zhenhua Guo, Omer Sunercan, Jun Ying, Thawan Kooburat, Suryadeep Biswal, Jun Chen, Kun Huang, Yatpang Cheung, Yiding Zhou, Kaushik Veeraraghavan, Biren Damani
2021Year
18Citations
21Top-tier citations
Abstract
Sharding is widely used to scale an application. Despite a decade of effort to build generic sharding frameworks that can be reused across different applications, the extent of their success remains unclear. We attempt to answer a fundamental question: what barriers prevent a sharding framework from getting adopted by the majority of sharded applications?
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Cited by top-tier papers21
- MAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at HyperscaleArnab Choudhury, Yang Wang, Tuomas Pelkonen, Kutta Srinivasan et al.OSDI 2024 · 39 citations
- Ekko: A Large-Scale Deep Learning Recommender System with Low-Latency Model UpdateChijun Sima, Yao Fu, Man-Kit Sit, Liyi Guo et al.OSDI 2022 · 31 citations
- Replicating Persistent Memory Key-Value Stores with Efficient RDMA AbstractionQing Wang, Youyou Lu, Jing Wang, Jiwu ShuOSDI 2023 · 14 citations
- ServiceLab: Preventing Tiny Performance Regressions at Hyperscale through Pre-Production TestingMike Chow, Yang Wang, William Wang, Ayichew Hailu et al.OSDI 2024 · 11 citations
- Parallelism-Optimizing Data Placement for Faster Data-Parallel ComputationsNirvik Baruah, Peter Kraft, Fiodar Kazhamiaka, Peter Bailis et al.VLDB 2023 · 9 citations
Related papers
- Areopagus: Asynchronous Sharding Blockchain Tolerating Corrupted Shards with Optimized OverheadYizhong Liu, Andi Liu, Yuxuan Hu, Jin Dong et al.CCS 2026
- ByShard: Sharding in a Byzantine EnvironmentJelle Hellings, Mohammad SadoghiVLDB 2021 · 103 citations
- Optimizing Resource Allocation in Hyperscale Datacenters: Scalability, Usability, and ExperiencesNeeraj Kumar, Pol Mauri Ruiz, Vijay Menon, Igor Kabiljo et al.OSDI 2024 · 6 citations
- Poros: Secure and Highly Parallel Sharding Blockchain with Optimized Cross-Shard Transaction ProcessingZhuocheng Pan, Jianwei Liu, Andi Liu, Yuxuan Hu et al.CCS 2026
- Yield Not Thy CoreAchilles Benetopoulos, Peter Alvaro, Andi Quinn, Robert SouléEuroSys 2026
