SkyPilot: An Intercloud Broker for Sky Computing
Zongheng Yang, Zhanghao Wu, Michael Luo, Wei-Lin Chiang, Romil Bhardwaj, Woosuk Kwon, Siyuan Zhuang, Frank Sifei Luan, Gautam Mittal, Scott Shenker, Ion Stoica
Abstract
To comply with the increasing number of government regulations about data placement and processing, and to protect themselves against major cloud outages, many users want the ability to easily migrate their workloads between clouds. In this paper we propose doing so not by imposing uniform and comprehensive standards, but by creating a fine-grained two-sided market via an intercloud broker. These brokers will allow users to view the cloud ecosystem not just as a collection of individual and largely incompatible clouds but as a more integrated Sky of Computing. We describe the design and implementation of an intercloud broker, named SkyPilot, evaluate its benefits, and report on its real-world usage.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 62b8620e-bfcd-45ca-9f86-414743323936Cited by top-tier papers33
- SpotServe: Serving Generative Large Language Models on Preemptible InstancesXupeng Miao, Chunan Shi, Jiangfei Duan, Xiaoli Xi et al.ASPLOS 2024 · 71 citations
- Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible InstancesJiangfei Duan, Ziang Song, Xupeng Miao, Xiaoli Xi et al.NSDI 2024 · 54 citations
- The Emergence of Reproducibility and Consistency in Diffusion ModelsHuijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo et al.ICML 2024 · 51 citations
- HexGen: Generative Inference of Large Language Model over Heterogeneous EnvironmentYouhe Jiang, Ran Yan, Xiaozhe Yao, Yang Zhou et al.ICML 2024 · 46 citations
- Can't Be Late: Optimizing Spot Instance Savings under DeadlinesZhanghao Wu, Wei-Lin Chiang, Ziming Mao, Zongheng Yang et al.NSDI 2024 · 40 citations
Builds on3
- ORION and the Three Rights: Sizing, Bundling, and Prewarming for Serverless DAGsAshraf Mahgoub, Edgardo Barsallo Yi, Karthick Shankar, Sameh Elnikety et al.OSDI 2022 · 111 citations
- Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep LearningLianmin Zheng, Zhuohan Li, Hao Zhang, Yonghao Zhuang et al.OSDI 2022 · 75 citations
- End-to-end Optimization of Machine Learning Prediction QueriesKwanghyun Park, Karla Saur, Dalitso Banda, Rathijit Sen et al.SIGMOD 2022 · 50 citations
Related papers
- Skyplane: Optimizing Transfer Cost and Throughput Using Cloud-Aware OverlaysParas Jain, Sam Kumar, Sarah Wooders, Shishir G. Patil et al.NSDI 2023 · 66 citations
- Enabling Live Migration of Containerized Applications Across CloudsThad Benjaponpitak, Meatasit Karakate, Kunwadee SripanidkulchaiINFOCOM 2020 · 53 citations
- SkyStore: Cost-Optimized Object Storage Across Regions and CloudsShu Liu, Xiangxi Mo, Moshik Hershcovitch, Henric Zhang et al.VLDB 2025 · 4 citations
- Meteor: High-Performance Control Message Delivery for Large-Scale CloudsGongming Zhao, Baoqing Wang, Min Chen, Hongli Xu et al.WWW 2026
- Invisinets: Removing Networking from Cloud NetworksSarah McClure, Zeke Medley, Deepak Bansal, Karthick Jayaraman et al.NSDI 2023
