C2F: Enabling Context-Aware Edge-Cloud Collaborative Inference for Foundation Models

Transformer-based foundation models (FMs) excel in diverse domains but struggle with high computation costs, hin-dering effective inference on resource-constrained edge devices. Edge-cloud collaborative inference offers a promising paradigm, but existing methods either neglect the contexts (computing resources and data environments) of edge devices or incur high transmission overheads when applied to FMs. In contrast, we propose C2F, a novel context-aware edge-cloud collaborative inference method for FMs, enabling open-set learning with low latency and high accuracy. In C2F, a context-aware model customization module is utilized to customize a small model (SM) from the FM based on the context, subsequently deployed on the designated edge device. During inference, an adaptive inference module is employed to determine whether to query the FM according to local results of the SM; if so, it transmits only vital data patches, effectively reducing transmission costs and end-to-end latency. Extensive experimental results demonstrate that C2F achieves efficient inference for FMs, enhancing accuracy by 3.1-30.3% and reducing end-to-end latency by 1.32-8.75× compared to various baselines.