Developmental Federated Tuning: A Cognitive-Inspired Paradigm for Efficient LLM Adaptation

Federated fine-tuning enables Large Language Models (LLMs) to adapt to downstream tasks while preserving data privacy, but its resource-intensive nature severely limits deployment on edge devices. In this paper, we introduce Developmental Federated Tuning (DEVFT), a resource-efficient approach inspired by cognitive development that progressively builds a powerful LLM from a compact foundation. DEVFT decomposes the fine-tuning process into developmental stages, each optimizing a submodel with increasing parameter capacity. Knowledge acquired in earlier stages is transferred to subsequent submodels, providing optimized initialization parameters that prevent convergence to local minima and accelerate training. This paradigm mirrors human learning, gradually constructing a comprehensive knowledge structure while refining existing skills. To efficiently build stage-specific submodels, DEVFT introduces deconflictionguided layer grouping and differential-based layer fusion to distill essential information and construct representative layers. Evaluations across multiple benchmarks demonstrate that DEVFT significantly outperforms state-of-the-art methods, achieving up to 4.59× faster convergence, 10.67× reduction in communication overhead, and 9.07% average performance improvement, while maintaining compatibility with existing federated fine-tuning approaches. * Equal Contribution. † Corresponding Authors. BACKGROUND AND MOTIVATION EXISTING PARAMETER-EFFICIENT FEDERATED FINE-TUNING Parameter-efficient federated fine-tuning presents a compelling strategy to mitigate resource demands in distributed learning by freezing most pre-trained model parameters and updating only a small, task-specific subset (Wu et al., 2025d). These methods generally fall into the following categories. Prompt-based techniques (Guo et al., 2023; Yang et al., 2023; Su et al., 2024) utilize carefully designed soft prompts to guide model behavior without altering the pre-trained weights. Adapter-