Time and Cost-Efficient Cloud Data Transmission based on Serverless Computing Compression

Nowadays, there exists a lot of cross-region data transmission demand on cloud. It is promising to use serverless computing for compressing data to save the transmission data amount. However, it is challenging to estimate the data transmission time and monetary cost with serverless compression. In addition, minimizing the data transmission cost is non-trivial due to enormous parameter space and joint optimization. This paper focuses on this problem and makes the following contributions: (1) We propose empirical data transmission time and monetary cost models based on serverless compression. (2) For single-task cloud data transmission, we propose two efficient parameter search methods based on Sequential Quadratic Programming (SQP ) and Eliminate then Divide and Conquer (EDC), which are theoretically proven with error upper bounds. (3) Furthermore, for multi-task cloud data transmission, a parameter search method based on dynamic programming and numerical computation is proposed to reduce the algorithm complexity from exponential to linear complexity. We have implemented the entire actual system and evaluated it with various workloads and application cases on the real-world AWS serverless computing platform. Experimental results on cross-region public cloud show that the proposed approach can improve the parameter search efficiency by more than 3× compared with the state-of-art parameter search methods and achieves better parameter quality. Compared with other competing cloud data transmission approaches, our approach is able to achieve higher time efficiency and lower monetary cost.