Multi-Tier Multi-Node Scheduling of LLM for Collaborative AI Computing

Large Language Models (LLMs) have attracted growing attention owing to their advanced capability in under-standing and reacting to instructions. While they are experiencing wide deployment in the multi-tier cloud-edge architecture, their performance is severely constrained by the network capacity, and how to schedule efficient data flow for performance maximum poses significant challenges. Towards that, this paper establishes a comprehensive system model and, for the first time, formulates the efficient LLM scheduling problem in the multi-tier cloud-edge network. Given its non-convexness, we propose the Multi-tier Multi-node Scheduling of LLM (MMSL) algorithm for Collabo-rative AI Computing, a two-stage scheduling framework designed to optimize LLM inference in multi-tier cloud-edge networks. Initially, the inter-tier LLM automated decoupling and partitioning phase employs integer linear programming to allocate model size and computing demands efficiently. Subsequently, the intra-tier LLM task scheduling algorithm, leveraging GNN, identifies optimal scheduling nodes within each tier by evaluating resource utilization and network conditions. Extensive evaluations show that our solution significantly outperforms traditional scheduling methods by 9.1 %- 26.3% throughput improvement.