Building an efficient key-value store in a flexible address space
Chen Chen, Wenshao Zhong, Xingbo Wu
Abstract
Data management applications store their data using structured files in which data are usually sorted to serve indexing and queries. However, in-place insertions and removals of data are not naturally supported in a file's address space. To avoid repeatedly rewriting existing data in a sorted file to admit changes in place, applications usually employ extra layers of indirections, such as mapping tables and logs, to admit changes out of place. However, this approach leads to increased access cost and excessive complexity.
This paper presents a novel storage abstraction that provides a flexible address space, where in-place updates of arbitrary-sized data, such as insertions and removals, can be performed efficiently. With these mechanisms, applications can manage sorted data in a linear address space with minimal complexity. Extensive evaluations show that a keyvalue store built on top of it can achieve high performance and efficiency with a simple implementation.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on8
- A large scale analysis of hundreds of in-memory cache clusters at TwitterJuncheng Yang, Yao Yue, K. V. RashmiOSDI 2020 · 245 citations
- MatrixKV: Reducing Write Stalls and Write Amplification in LSM-tree Based KV Stores with Matrix Container in NVMTing Yao, Yiwen Zhang, Jiguang Wan, Qiu Cui et al.USENIX ATC 2020 · 186 citations
- SpanDB: A Fast, Cost-Effective LSM-tree Based KV Store on Hybrid StorageHao Chen, Chaoyi Ruan, Cheng Li, Xiaosong Ma et al.FAST 2021 · 120 citations
- SplinterDB: Closing the Bandwidth Gap for NVMe Key-Value StoresAlexander Conway, Abhishek Gupta, Vijay Chidambaram, Martin Farach-Colton et al.USENIX ATC 2020 · 90 citations
- Viper: An Efficient Hybrid PMem-DRAM Key-Value StoreLawrence Benson, Hendrik Makait, Tilmann RablVLDB 2021 · 86 citations
Related papers
- WipDB: A Write-in-place Key-value Store that Mimics Bucket SortXingsheng Zhao, Song Jiang, Xingbo WuICDE 2021 · 17 citations
- Hardware-Based Address-Centric Acceleration of Key-Value StoreChencheng Ye, Yuanchao Xu, Xipeng Shen, Xiaofei Liao et al.HPCA 2021 · 6 citations
- REMIX: Efficient Range Query for LSM-treesWenshao Zhong, Chen Chen, Xingbo Wu, Song JiangFAST 2021 · 67 citations
- Differentiated Key-Value Storage Management for Balanced I/O PerformanceYongkun Li, Zhen Liu, Patrick P. C. Lee, Jiayu Wu et al.USENIX ATC 2021 · 79 citations
- Enhancing LSM-Tree Key-Value Stores for Read-Modify-Writes via Key-Delta SeparationJinhong Li, Yanjing Ren, Shujie Han, Patrick P. C. LeeICDE 2024 · 6 citations
