Secure Data Analytics in Apache Spark with Fine-grained Policy Enforcement and Isolated Execution

—Cloud based Spark platform is a tempting approach for sharing data, as it allows data users to easily analyze the data while the owners to efficiently share the large volume of data. However, the absence of a robust policy enforcement mechanism on Spark hinders the data owners from sharing their data due to the risk of private data breach. In this respect, we found that malicious data users and cloud managers can easily leak the data by constructing a policy violating physical plan, compromising the Spark libraries, or even compromising the Spark cluster itself. Nonetheless, current approaches fail to securely and generally enforce the policies on Spark, as they do not check the policies on physical plan level, and they do not protect the integrity of data analysis pipeline. This paper presents L APUTA 1 , a secure policy enforcement framework on Spark. Specifically, L APUTA designs a pattern matching based policy checking on the physical plans, which is generally applicable to Spark applications with more fine-grained policies. Then, L APUTA compartmentalizes Spark applications based on confidential computing, by which the entire data analysis pipeline is protected from the malicious data users and cloud managers. Meanwhile, L APUTA preserves the usability as the data users can run their Spark applications on L APUTA with minimal modification. We implemented L APUTA , and evaluated its security and performance aspects on TPC-H, Big Data benchmarks, and real world applications using ML models. The evaluation results demonstrated that L APUTA correctly blocks malicious Spark applications while imposing moderate performance overheads.