Lune

EMNLP Findings2025

Soteria: Language-Specific Functional Parameter Steering for Multilingual Safety Alignment

Somnath Banerjee, Sayan Layek, Pratyush Chatterjee, Animesh Mukherjee, Rima Hazra

Abstract

Ensuring consistent safety across multiple languages remains a significant challenge for large language models (LLMs). We introduce Soteria, a lightweight yet powerful strategy that locates and minimally adjusts the "functional heads" most responsible for harmful content generation in each language. By altering only a fraction of parameters, Soteria drastically reduces policy violations without sacrificing overall model performance, even in low-resource settings. To rigorously evaluate our approach, we also present XThreatBench, a specialized multilingual dataset capturing finegrained harmful behaviors drawn from real policy guidelines. Experiments with leading open-source LLMs (e.g., Llama, Qwen, Mistral) show that Soteria consistently improves safety metrics across high-, mid-, and lowresource languages. These findings highlight a promising path toward scalable, linguistically attuned, and ethically aligned LLMs worldwide. We release the source codes at: https: //github.com/neuralsentinel/soteria.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TruthfulQA: Measuring How Models Mimic Human Falsehoods cites Measuring Massive Multitask Language Understanding
  • Refusal in Language Models Is Mediated by a Single Direction cites Measuring Massive Multitask Language Understanding
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on30

  1. Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
  2. Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
  3. TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
  4. Refusal in Language Models Is Mediated by a Single DirectionAndy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka et al.NeurIPS 2024 · 1,166 citations
  5. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen et al.ICLR 2024 · 1,104 citations

Related papers

  1. The Heterogeneous Safety Impacts of Benign Multilingual Fine-TuningWill Hawkins, Kai Rawal, Jonathan Rystrøm, Stratis Tsirtsis et al.ICML 2026
  2. Multilingual Safety Alignment Via Sparse Weight EditingJiaming Liang, Zhaoxin Wang, Handing WangICML 2026 · 3 citations
  3. Multilingual Safety Alignment via Representation-Space SeparabilityDan Shi, Zhuowen Han, Deyi XiongICML 2026
  4. The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual ContextsLingfeng Shen, Weiting Tan, Sihao Chen, Yunmo Chen et al.ACL Findings 2024 · 25 citations
  5. Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety AlignmentYuyan Bu, Xiaohao Liu, ZhaoXing Ren, Yaodong Yang et al.ICLR 2026 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines