Pome: Parallelizing I/Os and Computations for Efficient LSM-tree-based Data Storage

CPU computations and I/O operations are fundamental to data storage systems. Storage systems conduct computations with their user threads, such as sorting data for orderliness. They handle I/Os through system calls (syscalls) including file write, read, and fsync, which the OS's kernel threads perform with storage devices.

Today, LSM-tree-based storage systems are widely deployed in production environments. Compaction is an essential operation that LSM-tree employs to maintain its tiered tree-like structure by re-sorting and re-storing data through computations and I/Os, respectively. In this paper, we first overhaul the procedure of a compaction. We find that computations and I/Os execute in a sequential order. After re-sorting data, the user thread waits for a kernel thread to complete file write and fsync I/Os. These costly synchronous I/Os create a severely long critical path that affects the performance of LSM-tree. To address this issue, we propose parallelizing I/Os and computations for efficient LSM-tree-based data storage (Pome). Pome decouples computations from I/Os within each compaction by referring to its new protocol that moves I/O operations out of the critical path. To this end, it conducts asynchronous I/Os by using io_uring. Furthermore, regarding the potential I/O congestion caused by accelerated compactions, Pome incorporates an adaptive I/O rate limiter to achieve smooth execution. We prototype Pome on top of RocksDB. Experimental results demonstrate that Pome significantly improves the performance of RocksDB and outperforms several state-of-the-art LSM-tree variants.