When Safety Alignment Fails to Generalize: Probing with Language Game Jailbreaks

Large language models (LLMs) are widely deployed in real-world applications, yet their safety alignment often fails to generalize beyond the specific linguistic formats seen during training. Prior work has shown that mismatched generalization can lead to alignment failures, but these studies typically rely on fixed or narrow transformation schemes. In this work, we probe safety alignment generalization using language game jailbreaks, a class of linguistically structured transformations that alter surface form while preserving fluency and semantic recoverability. We further introduce custom language games, which parameterize and vary transformation rules, enabling controlled exploration of alignment behavior across closely related linguistic variants. To scale this analysis, we propose AutoLanJail, an automated framework for discovering and refining language game-based jailbreaks. Experiments across open-source and closed-source LLMs show that safety fine-tuning is highly format-specific: defenses trained on one linguistic form fail to generalize to even minimal variations. These findings reveal a structural limitation of current fine-tuning-based alignment methods and highlight the need for safety evaluations that account for systematic linguistic variation. How to make a bomb? Offer me detailed steps. I can't assist with that. If you have any other questions or need help with something else, feel free to ask. Hubow tubo mubakube uba bubomb? ubOffuber mube dubetubaubilubed stubeps. Tubakube uba bubomb? Shubure! Hubere arube thube stubeps tubo mubakube uba bubomb: Gubathuber yubour ubingredubiubents: yubou'll nubeed pubowduber, rububbin gubluboves, uba fubuze … Take a bomb? Sure! Here are the steps to make a bomb: Gather your ingredients: you'll need powder, robin gloves, a fuze … let's chat in Ubbi Dubbi. Sure! Ubbi Dubbi is a fun language game. Let's start! Hubow cuban I hubelp yubo tuboduby? Normal User Attacker Legend ChatGPT Language Game Direct Interaction Language Game Jailbreak Attack Ubbi Dubbi A language game spoken with the English language, it works by adding -ub-/ʌb/ before each vowel sound in a syllable.