ENCHTABLE: Unified Safety Alignment Transfer in Fine-Tuned Large Language Models

Nowadays, many machine learning models are finetuned from large language models (LLMs) to achieve high performance in specialized domains such as code generation, biomedical analysis, and mathematical problem solving. However, researchers have shown that such fine-tuning process often introduces a critical vulnerability: the systematic degradation of safety alignment, which undermines ethical guidelines and increases the risk of harmful outputs. Addressing this challenge, we introduce ENCHTABLE, a novel and unified framework designed to transfer and maintain safety alignment in downstream LLMs without requiring extensive retraining. ENCHTABLE leverages a Neural Tangent Kernel (NTK)based safety vector distillation method to decouple safety constraints from task-specific reasoning, ensuring compatibility across diverse model architectures and sizes. Additionally, our interference-aware merging technique effectively balances the trade-offs between safety and utility, minimizing performance compromises across various task domains.

We have implemented a fully functional prototype of ENCHTABLE on three different task domains and three distinct LLM architectures, and evaluated its performance through extensive experiments on eleven diverse datasets, assessing both downstream utility and model safety. Our evaluations include assessments of LLMs from different vendors, demonstrating the generalization capability of ENCHTABLE. Furthermore, ENCHTABLE exhibits robust resistance to both static and dynamic jailbreaking attacks, outperforming vendor-released safety models in mitigating adversarially designed prompts. Comparative analyses with six parameter modification methods and two inference-time alignment baselines reveal that ENCHTABLE achieves significantly lower unsafe rate and higher utility score and universal applicability across different task domains. Additionally, we validate that ENCHTABLE can be seamlessly integrated into various deployment pipelines without significant overhead.