Balancing Repair Bandwidth and Sub-Packetization in Erasure-Coded Storage via Elastic Transformation

Erasure coding provides high fault-tolerant storage with significantly low redundancy overhead, at the expense of high repair bandwidth. While there exist access-optimal codes that theoretically minimize both the repair bandwidth and the amount of disk reads, they also incur a high sub-packetization level, thereby leading to non-sequential I/Os and degrading repair performance. We propose elastic transformation, a framework that transforms any base code into a new code with smaller repair bandwidth for all or a subset of nodes, such that it can be configured with a wide range of sub-packetization levels to limit the non-sequential I/O overhead. We prove the fault tolerance of elastic transformation and model numerically the repair performance with respect to a sub-packetization level. We further prototype and evaluate elastic transformation atop HDFS, and show how it reduces the single-block repair time of the base codes and access-optimal codes in a real network setting.

• We show how our elastic transformation reduces the repair bandwidth of various erasure codes, including RS codes [32], Azure's LRC [13], Hitchhiker [31], and HashTag [18]. • On the theoretical side, we prove the fault tolerance of elastic transformation, model the lower bound of repair bandwidth for a given sub-packetization level, and model the repair time subject to the bandwidth and I/O conditions. • Existing code transformation solutions [11], [12], [19], [20]

only focus on theoretical analysis, but do not consider empirical evaluation. To fill this void, we prototype elastic transformation based on OpenEC [23] and evaluate it atop Hadoop 3.0.0 HDFS [1]. Experiments on our prototype, called OpenEC-ET, show that it reduces the repair time of the base RS codes (without transformation) by up to 56.3% in low-bandwidth settings and reduces the repair time of the access-optimal MSR codes by up to 51.4% in highbandwidth settings. The source code of OpenEC-ET is at: http://adslab.cse.cuhk.edu.hk/software/openec-et.