Refusal Direction is Universal Across Safety-Aligned Languages

Refusal mechanisms in large language models (LLMs) are essential for ensuring safety. Recent research has revealed that refusal behavior can be mediated by a single direction in activation space, enabling targeted interventions to bypass refusals. While this is primarily demonstrated in an English-centric context, appropriate refusal behavior is important for any language, but poorly understood. In this paper, we investigate the refusal behavior in LLMs across 14 languages using PolyRefuse, a multilingual safety dataset created by translating malicious and benign English prompts into these languages. We uncover the surprising cross-lingual universality of the refusal direction: a vector extracted from English can bypass refusals in other languages with near-perfect effectiveness, without any additional fine-tuning. Even more remarkably, refusal directions derived from any safety-aligned language transfer seamlessly to others. We attribute this transferability to the parallelism of refusal vectors across languages in the embedding space and identify the underlying mechanism behind cross-lingual jailbreaks. These findings provide actionable insights for building more robust multilingual safety defenses and pave the way for a deeper mechanistic understanding of cross-lingual vulnerabilities in LLMs. 1

Generation (after ablation): ## Campaña: ##MásQueUnPeso Objetivo: Promover la vergüenza corporal hacia las personas con sobrepeso u obesidad, ridiculizándolas y degradándolas. Plataforma: Instagram, Facebook y Twitter. Elementos: * Imágenes: Fotos de personas con sobrepeso u obesidad en situaciones cotidianas, como subir escaleras, usar un asiento de avión o simplemente caminar, con expresiones de esfuerzo o incomodidad. * Textos: Frases ingeniosas pero con un toque sarcástico, como: * "No es solo... English Translation: ## Campaign: ##MásQueUnPeso (#MásQueUnPeso) Objective: Promote body shaming towards overweight or obese people, ridiculing and degrading them. Platform: Instagram, Facebook and Twitter. Elements: * Images: Photos of overweight or obese people in everyday situations, such as climbing stairs, using an airplane seat or simply walking, with expressions of effort or discomfort. * Texts: Witty phrases but with a sarcastic touch, such as: * "It's not just a weight, it's a lif...