Lune

ACL Findings2026

Multilingual Refusal Alignment for Safer Large Language Models

Aleksandra Krasnodebska, Wojciech Kusa, Aldo Lipani

Abstract

As Large Language Models (LLMs) are deployed globally, ensuring their safety and alignment across multiple languages becomes paramount. However, safety behaviors often vary unpredictably between languages, posing significant challenges for consistent and ethical AI. In this work, we systematically investigate the dynamics of multilingual alignment, exploring whether single-language alignment transfers cross-lingually, how language consistency is preserved during training, and the resulting trade-offs with general knowledge capabilities. We introduce RefusEU 1 , a novel refusal alignment dataset covering 12 European languages, including a dedicated test set for evaluating current state-of-the-art models. Our controlled Direct Preference Optimization (DPO) experiments provide two key insights: aligning models exclusively in English is insufficient to ensure cross-lingual safety, even for the same harm categories, whereas training on multilingual datasets can improve safety without degrading general performance, as measured by the Global MMLU benchmark.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Training language models to follow instructions with human feedback
  • Multilingual Jailbreak Challenges in Large Language Models cites Training language models to follow instructions with human feedback
  • Refusal in Language Models Is Mediated by a Single Direction cites Training language models to follow instructions with human feedback
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on12

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  3. Refusal in Language Models Is Mediated by a Single DirectionAndy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka et al.NeurIPS 2024 · 1,166 citations
  4. Rainbow Teaming: Open-Ended Generation of Diverse Adversarial PromptsMikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro et al.NeurIPS 2024 · 231 citations
  5. Multilingual Jailbreak Challenges in Large Language ModelsYue Deng, Wenxuan Zhang, Sinno Jialin Pan, Lidong BingICLR 2024 · 230 citations

Related papers

  1. MPO: Multilingual Safety Alignment via Reward Gap OptimizationWeixiang Zhao, Yulin Hu, Yang Deng, Tongtong Wu et al.ACL 2025 · 12 citations
  2. PLLuM-Align: Polish Preference Dataset for Large Language Model AlignmentKarolina Seweryn, Anna Kolos, Agnieszka Karlinska, Katarzyna Lorenc et al.EMNLP 2025
  3. Refusal Direction is Universal Across Safety-Aligned LanguagesXinpeng Wang, Mingyang Wang, Yihong Liu, Hinrich Schütze et al.NeurIPS 2025 · 39 citations
  4. The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce HarmAakanksha, Arash Ahmadian, Beyza Ermis, Seraphina Goldfarb-Tarrant et al.EMNLP 2024 · 3 citations
  5. Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety AlignmentYuyan Bu, Xiaohao Liu, ZhaoXing Ren, Yaodong Yang et al.ICLR 2026 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines