Lune

ACL Findings2025

COSMIC: Generalized Refusal Direction Identification in LLM Activations

Vincent Siu, Nicholas Crispino, Zihao Yu, Sam Pan, Zhun Wang, Yang Liu, Dawn Song, Chenguang Wang

Abstract

Large Language Models (LLMs) encode behaviors such as refusal within their activation space, yet identifying these behaviors remains a significant challenge. Existing methods often rely on predefined refusal templates detectable in output tokens or require manual analysis. We introduce COSMIC (Cosine Similarity Metrics for Inversion of Concepts), an automated framework for direction selection that identifies viable steering directions and target layers using cosine similarity-entirely independent of model outputs. COSMIC achieves steering performance comparable to prior methods without requiring assumptions about a model's refusal behavior, such as the presence of specific refusal tokens. It reliably identifies refusal directions in adversarial settings and weakly aligned models, and is capable of steering such models toward safer behavior with minimal increase in false refusals, demonstrating robustness across a wide range of alignment conditions. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LLMs Encode Harmfulness and Refusal Separately cites Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!
  • LLMs Encode Harmfulness and Refusal Separately cites Training language models to follow instructions with human feedback
  • LLMs Encode Harmfulness and Refusal Separately cites Refusal in Language Models Is Mediated by a Single Direction
  • TruthfulQA: Measuring How Models Mimic Human Falsehoods cites Training language models to follow instructions with human feedback
  • Training language models to follow instructions with human feedback cites TruthfulQA: Measuring How Models Mimic Human Falsehoods
  • Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! cites Training language models to follow instructions with human feedback
  • Dynamic Token Reweighting for Robust Vision-Language Models cites Refusal in Language Models Is Mediated by a Single Direction
  • Refusal in Language Models Is Mediated by a Single Direction cites Sparse Autoencoders Find Highly Interpretable Features in Language Models
  • Refusal in Language Models Is Mediated by a Single Direction cites Training language models to follow instructions with human feedback
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. LLMs Encode Harmfulness and Refusal SeparatelyJiachen Zhao, Jing Huang, Zhengxuan Wu, David Bau et al.NeurIPS 2025 · 93 citations
  2. ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following BehaviorWeikai Lu, Ziqian Zeng, Kehua Zhang, Haoran Li et al.CVPR 2026 · 6 citations
  3. Dynamic Token Reweighting for Robust Vision-Language ModelsTanqiu Jiang, Jiacheng Liang, Rongyi Zhu, Jiawei Zhou et al.CVPR 2026 · 2 citations

Builds on18

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
  3. Refusal in Language Models Is Mediated by a Single DirectionAndy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka et al.NeurIPS 2024 · 1,166 citations
  4. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen et al.ICLR 2024 · 1,104 citations
  5. Sparse Autoencoders Find Highly Interpretable Features in Language ModelsRobert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart et al.ICLR 2024 · 1,072 citations

Related papers

  1. The Geometry of Refusal in Large Language Models: Concept Cones and Representational IndependenceTom Wollschläger, Jannes Elstner, Simon Geisler, Vincent Cohen-Addad et al.ICML 2025
  2. Angular Steering: Behavior Control via Rotation in Activation SpaceMinh Hieu Vu, Tan M. NguyenNeurIPS 2025 · 53 citations
  3. SOM Directions Are Better than One: Multi-Directional Refusal Suppression in Language ModelsGiorgio Piras, Raffaele Mura, Fabio Brau, Luca Oneto et al.AAAI 2026 · 4 citations
  4. LayerNavigator: Finding Promising Intervention Layers for Efficient Activation Steering in Large Language ModelsHao Sun, Huailiang Peng, Qiong Dai, Xu Bai et al.NeurIPS 2025 · 4 citations
  5. Understanding Refusal in Language Models with Sparse AutoencodersWei Jie Yeo, Nirmalendu Prakash, Clement Neo, Ranjan Satapathy et al.EMNLP Findings 2025
Dusk over the sea between two cliffs drawn in fine vertical lines