Lune

EMNLP Findings2025

Understanding Refusal in Language Models with Sparse Autoencoders

Wei Jie Yeo, Nirmalendu Prakash, Clement Neo, Ranjan Satapathy, Roy Ka-Wei Lee, Erik Cambria

Abstract

Refusal is a key safety behavior in aligned language models, yet the internal mechanisms driving refusals remain opaque. In this work, we conduct a mechanistic study of refusal in instruction-tuned LLMs using sparse autoencoders to identify latent features that causally mediate refusal behaviors. We apply our method to two open-source chat models and intervene on refusal-related features to assess their influence on generation, validating their behavioral impact across multiple harmful datasets. This enables a fine-grained inspection of how refusal manifests at the activation level and addresses key research questions such as investigating upstream-downstream latent relationship and understanding the mechanisms of adversarial jailbreaking techniques. We also establish the usefulness of refusal features in enhancing generalization for linear probes to outof-distribution adversarial samples in classification tasks. We open source our code in https: //github.com/SenticNet/Refusal-SAE.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Locating and Editing Factual Associations in GPT cites Investigating Gender Bias in Language Models Using Causal Mediation Analysis
  • The Linear Representation Hypothesis and the Geometry of Large Language Models cites Locating and Editing Factual Associations in GPT
  • Do Sparse Autoencoders Identify Reasoning Features in Language Models? cites The Linear Representation Hypothesis and the Geometry of Large Language Models
  • Inference-Time Intervention: Eliciting Truthful Answers from a Language Model cites Locating and Editing Factual Associations in GPT
  • Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models cites HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Do Sparse Autoencoders Identify Reasoning Features in Language Models?George Ma, Zhongyuan Liang, Irene Y. Chen, Somayeh SojoudiICML 2026 · 9 citations
  2. Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language ModelsYanchen Yin, Dongqi Han, Linghui LiICML 2026 · 1 citation
  3. C2^{2}R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse AutoencodersHaoran Jin, Xiting Wang, Shijie Ren, Hong Xie et al.ICML 2026

Builds on9

  1. Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
  2. Inference-Time Intervention: Eliciting Truthful Answers from a Language ModelKenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister et al.NeurIPS 2023 · 1,549 citations
  3. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust RefusalMantas Mazeika, Long Phan, Xuwang Yin, Andy Zou et al.ICML 2024 · 1,031 citations
  4. Investigating Gender Bias in Language Models Using Causal Mediation AnalysisJesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian et al.NeurIPS 2020 · 851 citations
  5. The Linear Representation Hypothesis and the Geometry of Large Language ModelsKiho Park, Yo Joong Choe, Victor VeitchICML 2024 · 461 citations

Related papers

  1. Refusal in Language Models Is Mediated by a Single DirectionAndy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka et al.NeurIPS 2024 · 1,166 citations
  2. SOM Directions Are Better than One: Multi-Directional Refusal Suppression in Language ModelsGiorgio Piras, Raffaele Mura, Fabio Brau, Luca Oneto et al.AAAI 2026 · 4 citations
  3. The Geometry of Refusal in Large Language Models: Concept Cones and Representational IndependenceTom Wollschläger, Jannes Elstner, Simon Geisler, Vincent Cohen-Addad et al.ICML 2025
  4. The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety DirectionsWenbo Pan, Zhichao Liu, Qiguang Chen, Xiangyang Zhou et al.ICML 2025
  5. ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking AttackYein Park, Jungwoo Park, Jaewoo KangICLR 2026 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines