Steering Beyond the Support: Adversarial Training on Unsupervised Jailbroken Activation Simulation

Jailbreak prompts can trigger harmful completions from aligned LLMs. Accordingly, safety steering has been proposed as a family of test-time activation interventions that redirect jailbreak activations toward refusal while preserving benign utility. However, existing steering methods are fundamentally supervised and tied to a static, limited training set, whereas real jailbreaks evolve and are often out of distribution relative to the training set, leading to failures on unseen attacks. In this paper, we tackle this failure by developing a zero-shot defense. Based on unsupervised latent direction discovery, we directly simulate jailbroken activations without any knowledge of jailbreak strategy. To build a defense mechanism on top of this simulation, we propose a bi-level adversarial training framework. In the inner step, we simulate diverse jailbroken activations by extrapolating from refusal-state harmful-request activations via unsupervised latent direction discovery. In the outer step, we train a potential-induced steering field to push these adversarial jailbroken states into refusal regions while leaving benign activations unchanged. Across three LLMs and six classical jailbreak families, our method achieves strong defense with attack success rates mostly below 5%. We further analyze the increasing subspace coverage of our simulated jailbroken activations over real jailbreaks throughout training, which helps explain the increasing robustness of our defense mechanism.