How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs

Most traditional AI safety research views models as machines and centers on algorithmfocused attacks developed by security experts. As large language models (LLMs) become increasingly common and competent, non-expert users can also impose risks during daily interactions. Observing this, we shift the perspective, by treating LLMs as human-like communicators to examine the interplay between everyday language interaction and AI safety. Specifically, we study how to persuade LLMs to jailbreak them. First, we propose a persuasion taxonomy derived from decades of social science research. Then, we apply the taxonomy to automatically generate persuasive adversarial prompts (PAP) to jailbreak LLMs. Results show that persuasion significantly increases the jailbreak risk across all risk categories: PAP consistently achieves an attack success rate of over 92% on Llama-2-7b-Chat, GPT-3.5, and GPT-4 in 10 trials, surpassing recent algorithm-focused attacks. On the defense side, we explore various mechanisms against PAP, find a significant gap in existing defenses, and advocate for more fundamental solutions for AI safety 1 . * Lead authors. Corresponding Y. Zeng, W. Shi, R. Jia † Co-supervised the project, listed alphabetically. The work was done while W.S. was at Stanford. 1 We have informed Meta and OpenAI of our findings. For safety concerns, we only publicly release our persuasion taxonomy at https://github.com/CHATS-lab/ persuasive_jailbreaker . Researchers can apply for the jailbreaking data upon review.