LaCo: Layer-wise Compensation for Pruned Large Language Models

Pruning is essential for the efficient deployment of Large Language Models (LLMs); however, it causes severe performance degradation due to the structural distortion induced by sparsity. Existing recovery strategies, such as LoRA, predominantly employ global finetuning, often overlooking the mechanistic root of this degradation: the layer-wise accumulation and amplification of local errors. To address this limitation, we propose LaCo (Layerwise Compensation), a framework that reorients the recovery paradigm from global adaptation to hierarchical representation alignment. By sequentially optimizing each layer to reconstruct the model's hidden states, LaCo effectively intercepts the error propagation chain at its source. Extensive experiments demonstrate that LaCo surpasses parameter-efficient baselines in both perplexity reduction and zeroshot reasoning. Notably, it reduces recoverytime memory usage to approximately 1/7 of the baseline and requires only 2,048 unlabeled samples to match a LoRA model trained on 50k examples-achieving a ∼ 25× improvement in data efficiency. 0 5 10 15 20 25 30 Layer Index 10 5 10 4 10 3 10 2 10 1 10 0 10 1 MSE Distance (Log Scale) Layer-wise Error Accumulatio Intercepting Error Chain Impact of Error Propagation at 70% Sparsity Pruned (No Compensation) LaCo (Ours) Mitigated Error Accumulation (a) Error Accumulation 0 5 10 15 20 25 30 Layer Index