Lune

EMNLP Findings2025

SwiftPrune: Hessian-Free Weight Pruning for Large Language Models

Yuhan Kang, Yang Shi, Mei Wen, Jun He, Jianchao Yang, Zeyu Xue, Jing Feng, Xinwang Liu

Abstract

Post-training pruning, as one of the key techniques for compressing large language models (LLMs), plays a vital role in lightweight model deployment and model sparsity. However, current mainstream pruning methods dependent on the Hessian matrix face significant limitations in both pruning speed and practical effectiveness due to the computationally intensive nature of second-order derivative calculations. This paper presents SwiftPrune, a novel Hessian-free weight pruning method that achieves hardware-efficient model compression through two key innovations: 1) SwiftPrune eliminates the need for computationally intensive Hessian matrix calculations by introducing a contribution-based weight metric, which evaluates the importance of weights without relying on second-order derivatives. 2) we employ the Exponentially Weighted Moving Average (EWMA) technique to bypass weight sorting, enabling the selection of weights that contribute most to LLMs accuracy and further reducing time complexity. Our approach is extended to support structured sparsity pruning, facilitating efficient execution on modern hardware accelerators. We validate the SwiftPrune on three LLMs (namely LLaMA2, LLaMA3, and Pythia), demonstrating that it significantly enhances compression performance. The experimental findings reveal that SwiftPrune completes the pruning process within seconds, achieving an average speedup of 12.29× (up to 56.02×) over existing SOTA approaches.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • SparseGPT: Massive Language Models Can be Accurately Pruned in One-Shot cites Up or Down? Adaptive Rounding for Post-Training Quantization
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on20

  1. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  2. PIQA: Reasoning about Physical Commonsense in Natural LanguageYonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao et al.AAAI 2020 · 2,916 citations
  3. Pythia: A Suite for Analyzing Large Language Models Across Training and ScalingStella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley et al.ICML 2023 · 1,822 citations
  4. SparseGPT: Massive Language Models Can be Accurately Pruned in One-ShotElias Frantar, Dan AlistarhICML 2023 · 1,240 citations
  5. Up or Down? Adaptive Rounding for Post-Training QuantizationMarkus Nagel, Rana Ali Amjad, Mart van Baalen, Christos Louizos et al.ICML 2020 · 816 citations

Related papers

  1. Plug-and-Play: An Efficient Post-training Pruning Method for Large Language ModelsYingtao Zhang, Haoli Bai, Haokun Lin, Jialin Zhao et al.ICLR 2024 · 72 citations
  2. LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-TuningMingyang Zhang, Hao Chen, Chunhua Shen, Zhen Yang et al.ACL Findings 2024 · 64 citations
  3. DLP: Dynamic Layerwise Pruning in Large Language ModelsYuli Chen, Bo Cheng, Jiale Han, Yingying Zhang et al.ICML 2025
  4. Pruning Foundation Models for High Accuracy without RetrainingPu Zhao, Fei Sun, Xuan Shen, Pinrui Yu et al.EMNLP Findings 2024 · 13 citations
  5. Lightweight Haar Wavelet Subband Pruning for LLMsJiang Li, Pengfei Cao, Chenxi Zhou, Tian Lan et al.ACL Findings 2026
Dusk over the sea between two cliffs drawn in fine vertical lines