A Structure-Agnostic Co-Tuning Framework for LLMs and SLMs in Cloud-Edge Systems
Yuze Liu, Yunhan Wang, Tiehua Zhang, Zhishu Shen, Cheng Peng, Libing Wu, Feng Xia, Jiong Jin
Abstract
The surge in intelligent applications driven by large language models (LLMs) has made it increasingly difficult for bandwidth-limited cloud servers to process extensive LLM workloads in real time without compromising user data privacy. To solve these problems, recent research has focused on constructing cloud-edge consortia that integrate server-based LLM with small language models (SLMs) on mobile edge devices. Furthermore, designing collaborative training mechanisms within such consortia to enhance inference performance has emerged as a promising research direction. However, the cross-domain deployment of SLMs, coupled with structural heterogeneity in SLMs architectures, poses significant challenges to enhancing model performance. To this end, we propose Co-PLMs, a novel co-tuning framework for collaborative training of large and small language models, which integrates the process of structure-agnostic mutual learning to realize knowledge exchange between the heterogeneous language models. This framework employs distilled proxy models (DPMs) as bridges to enable collaborative training between the heterogeneous server-based LLM and on-device SLMs, while preserving the domain-specific insights of each device. The experimental results show that Co-PLMs outperforms state-of-the-art methods, achieving average increases of 5.38% in Rouge-L and 4.88% in EM. Our code has been released at https://github.com/papercode-DFL/Co-PLMs .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- FedPETuning: When Federated Learning Meets the Parameter-Efficient Tuning Methods of Pre-trained Language Models cites LoRA: Low-Rank Adaptation of Large Language Models
- Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning cites Measuring Massive Multitask Language Understanding
Builds on10
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
- Sheared LLaMA: Accelerating Language Model Pre-training via Structured PruningMengzhou Xia, Tianyu Gao, Zhiyuan Zeng, Danqi ChenICLR 2024 · 453 citations
- Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP TasksYizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi et al.EMNLP 2022 · 238 citations
- FedPETuning: When Federated Learning Meets the Parameter-Efficient Tuning Methods of Pre-trained Language ModelsZhuo Zhang, Yuanhang Yang, Yong Dai, Qifan Wang et al.ACL Findings 2023 · 98 citations
Related papers
- CoTuning: A Large-Small Model Collaborating Distillation Framework for Better Model GeneralizationZimo Liu, Kangjun Liu, Mingyue Guo, Shiliang Zhang et al.ACM MM 2024 · 3 citations
- Co-training and Co-distillation for Quality Improvement and Compression of Language ModelsHayeon Lee, Rui Hou, Jongpil Kim, Davis Liang et al.EMNLP Findings 2023 · 1 citation
- CoEdge-RAG: Optimizing Hierarchical Scheduling for Retrieval-Augmented LLMs in Collaborative Edge ComputingGuihang Hong, Tao Ouyang, Kongyange Zhao, Zhi Zhou et al.RTSS 2025 · 4 citations
- Collaboration of Large Language Models and Small Recommendation Models for Device-Cloud RecommendationZheqi Lv, Tianyu Zhan, Wenjie Wang, Xinyu Lin et al.KDD 2025 · 4 citations
- CLONE: Customizing LLMs for Efficient Latency-Aware Inference at the EdgeChunlin Tian, Xinpeng Qin, Kahou Tam, Li Li et al.USENIX ATC 2025 · 41 citations
