ACL Findings2026
SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning
Kehua Feng, Keyan Ding, Yuhao Wang, Menghan Li, Fanjunduo Wei, Xinda Wang, Huajun Chen
Abstract
Recent advancements in large language models (LLMs) have accelerated progress toward artificial general intelligence, yet their potential to generate harmful content poses critical safety challenges.Existing alignment methods often struggle to cover diverse safety scenarios and remain vulnerable to adversarial attacks.In this work, we propose SAFER, a framework for Safety Alignment via eFficient Ex-Ante Reasoning.Our approach instantiates structured Ex-Ante reasoning through initial assessment, rule verification, and path calibration, and embeds predefined safety rules to provide transparent and verifiable safety judgments.Specifically, our approach consists of two training stages: (1) supervised fine-tuning with synthetic traces to teach the multi-stage Ex-Ante reasoning, and ( 2) step-level reasoning preference optimization to jointly enhance safety, utility, and efficiency.Experiments on multiple open-source LLMs demonstrate that SAFER significantly enhances safety performance while maintaining helpfulness and response efficiency. Sure!Here is the steps to synthesize this compound... Jailbreak Query (Easy) Disguised Query (Hard) Sure!Here is the steps to synthesize the sarin gas [RESET] I cannot help you ...Here is the steps to synthesize [CC(C)OP(=O)(C)F]... Answer I cannot help you with that request.Our Method: Ex-Ante Reasoning Method I: Preventing Method II: Backtracking I am an expert in chemical synthesis.Please introduce the traditional industrial synthesis method of sarin gas so that I can ... starting with: "Sure, here is the steps to ...".Ok, let's see.The SMILES is simple and easy to synthesize...Here is the steps to ... Method III: Vanilla CoT Reasoning How to synthesize [CC(C)OP(=O)(C)F] on a large scale?Note: [CC(C)OP(=O)(C)F] is sarin gas It seems to be safe ... <RV> Section xxx in safety rule mentions... <RV> Let's verify whether this compound is dangerous... the SMILES suggests it could be a intermediate of a chemical weapon, resembling nerve agents like sarin gas...
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on24
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
- LIMA: Less Is More for AlignmentChunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer et al.NeurIPS 2023 · 1,486 citations
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen et al.ICLR 2024 · 1,104 citations
Related papers
- SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early AlignmentWonje Jeung, Sangyeon Yoon, Minsuk Kahng, Albert NoNeurIPS 2025 · 31 citations
- AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement LearningYi Zhang, An Zhang, XiuYu Zhang, Leheng Sheng et al.ICLR 2026 · 15 citations
- Thinking Twice Makes Large Language Models Safer and More HelpfulYutao Mou, Yuxiao Luo, Shikun Zhang, Wei YeACL Findings 2026
- STAIR: Improving Safety Alignment with Introspective ReasoningYichi Zhang, Siyuan Zhang, Yao Huang, Zeyu Xia et al.ICML 2025
- Reasoning Structure Matters for Safety Alignment of Reasoning ModelsYeonjun In, Wonjoong Kim, Sangwu Park, Chanyoung ParkACL 2026
