CSPLoRA: Confidence-Guided Structure Planning for Low-Rank Adaptation

Low-Rank Adaptation (LoRA) has become the de facto paradigm for parameter-efficient fine-tuning, with its effectiveness critically influenced by rank allocation across modules. However, existing approaches face a fundamental dilemma: uniform allocation ignores module heterogeneity, while adaptive methods introduce expensive training overhead or lack reusability across configurations. We propose CSPLoRA (Confidence-guided Structural Planning for LoRA), a decoupled framework that reweights probe samples by prediction uncertainty to obtain more discriminative module importance estimates. The key insight is that hard samples---those the model struggles with---provide more informative gradient signals for identifying critical modules than easy samples. For a fixed task-model pair, the resulting structural priors can be reused across compatible rank budgets and LoRA backends, supporting a practical "probe once, deploy everywhere" workflow. Experiments on GLUE, commonsense reasoning, and arithmetic tasks show that CSPLoRA improves over uniform LoRA on average (+1.25 points on LLaMA-2-7B commonsense reasoning) while maintaining comparable parameters, with the planned rank structure reusable across compatible LoRA variants.