Lune

ACL Findings2023

Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias

Zhiyuan Zhang, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun

Abstract

Pre-trained Language Models (PLMs) may be poisonous with backdoors or bias injected by the suspicious attacker during the fine-tuning process. A core challenge of purifying potentially poisonous PLMs is precisely finding poisonous dimensions. To settle this issue, we propose the Fine-purifying approach, which utilizes the diffusion theory to study the dynamic process of fine-tuning for finding potentially poisonous dimensions. According to the relationship between parameter drifts and Hessians of different dimensions, we can detect poisonous dimensions with abnormal dynamics, purify them by resetting them to clean pretrained weights, and then fine-tune the purified weights on a small clean dataset. To the best of our knowledge, we are the first to study the dynamics guided by the diffusion theory for safety or defense purposes. Experimental results validate the effectiveness of Fine-purifying even with a small clean dataset.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Don't Shift the Trigger: Robust Gradient Ascent for Backdoor Unlearning cites Weight Poisoning Attacks on Pretrained Models
  • Unmasking Backdoors: An Explainable Defense via Gradient-Attention Anomaly Scoring for Pre-trained Language Models cites Weight Poisoning Attacks on Pretrained Models
  • Unmasking Backdoors: An Explainable Defense via Gradient-Attention Anomaly Scoring for Pre-trained Language Models cites Defense against Backdoor Attack on Pre-trained Language Models via Head Pruning and Attention Normalization
  • Acquiring Clean Language Models from Backdoor Poisoned Datasets by Downscaling Frequency Space cites Weight Poisoning Attacks on Pretrained Models
  • Diffusion Models Beat GANs on Image Synthesis cites Language Models are Few-Shot Learners
  • Diffusion Models Beat GANs on Image Synthesis cites Denoising Diffusion Probabilistic Models
  • Weight Poisoning Attacks on Pretrained Models cites Latent Backdoor Attacks on Deep Neural Networks
  • Defense against Backdoor Attack on Pre-trained Language Models via Head Pruning and Attention Normalization cites Weight Poisoning Attacks on Pretrained Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Defense against Backdoor Attack on Pre-trained Language Models via Head Pruning and Attention NormalizationXingyi Zhao, Depeng Xu, Shuhan YuanICML 2024 · 17 citations
  2. Unmasking Backdoors: An Explainable Defense via Gradient-Attention Anomaly Scoring for Pre-trained Language ModelsAnindya Sundar Das, Kangjie Chen, Monowar BhuyanICLR 2026 · 4 citations
  3. Don't Shift the Trigger: Robust Gradient Ascent for Backdoor UnlearningXingyi Zhao, Tian Xie, Xiaojun Qi, Depeng Xu et al.ICLR 2026
  4. Acquiring Clean Language Models from Backdoor Poisoned Datasets by Downscaling Frequency SpaceZongru Wu, Zhuosheng Zhang, Pengzhou Cheng, Gongshen LiuACL 2024

Builds on13

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
  3. Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations
  4. Latent Backdoor Attacks on Deep Neural NetworksYuanshun Yao, Huiying Li, Haitao Zheng, Ben Y. ZhaoCCS 2019 · 465 citations
  5. Weight Poisoning Attacks on Pretrained ModelsKeita Kurita, Paul Michel, Graham NeubigACL 2020 · 312 citations

Related papers

  1. Fine-mixing: Mitigating Backdoors in Fine-tuned Language ModelsZhiyuan Zhang, Lingjuan Lyu, Xingjun Ma, Chenguang Wang et al.EMNLP Findings 2022 · 16 citations
  2. PurMM: Attention-Guided Test-Time Backdoor Purification in Multimodal Large Language ModelsWenzheng Jiang, Ke Liang, Xuankun Rong, Jingxuan Zhou et al.AAAI 2026
  3. DiSec: Mitigating Backdoors in Pre-trained Language Models via Disentanglement of Adversarial Weights for Secure Fine-TuningSunanda Das, Qinghua LiACL Findings 2026
  4. Backdoor Mitigation by Distance-Driven DetoxificationShaokui Wei, Jiayin Liu, Hongyuan ZhaICCV 2025 · 1 citation
  5. Persistent Pre-training Poisoning of LLMsYiming Zhang, Javier Rando, Ivan Evtimov, Jianfeng Chi et al.ICLR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines