EMNLP Findings2023
InstructSafety: A Unified Framework for Building Multidimensional and Explainable Safety Detector through Instruction Tuning
Zhexin Zhang, Jiale Cheng, Hao Sun, Jiawen Deng, Minlie Huang
Abstract
Safety detection has been an increasingly important topic in recent years and it has become even more necessary to develop reliable safety detection systems with the rapid development of large language models. However, currently available safety detection systems have limitations in terms of their versatility and interpretability. In this paper, we first introduce INSTRUCTSAFETY, a safety detection framework that unifies 7 common sub-tasks for safety detection. These tasks are unified into a similar form through different instructions. We then conduct a comprehensive survey of existing safety detection datasets and process 39 human-annotated datasets for instruction tuning. We also construct adversarial samples to enhance the model's robustness. After fine-tuning Flan-T5 on the collected data, we have developed Safety-Flan-T5, a multidimensional and explainable safety detector. We conduct comprehensive experiments on a variety of datasets and tasks, and demonstrate the strong performance of Safety-Flan-T5 in comparison to supervised baselines and served APIs (Perspective API, ChatGPT and In-structGPT). Our Github repository is at https: //github.com/thu-coai/InstructSafety.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification cites Detoxifying Large Language Models via Knowledge Editing
- Detoxification for LLM: From Dataset Itself cites Detoxifying Large Language Models via Knowledge Editing
Cited by top-tier papers3
- Projecting Out the Malice: A Global Subspace Approach to LLM DetoxificationZenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang et al.ACL 2026 · 2 citations
- Detoxifying Large Language Models via Knowledge EditingMengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi et al.ACL 2024
- Detoxification for LLM: From Dataset ItselfWei Shao, Yihang Wang, Gao yu Zhu, Ziqiang Cheng et al.ACL 2026
Builds on27
- Multitask Prompted Training Enables Zero-Shot Task GeneralizationVictor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach et al.ICLR 2022 · 1,976 citations
- The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural NetworksNicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos et al.USENIX Security 2019 · 1,386 citations
- Aligning AI With Shared Human ValuesDan Hendrycks, Collin Burns, Steven Basart, Andrew Critch et al.ICLR 2021 · 878 citations
- Privacy Risks of General-Purpose Language ModelsXudong Pan, Mi Zhang, Shouling Ji, Min YangS&P 2020 · 291 citations
- Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP TasksYizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi et al.EMNLP 2022 · 238 citations
Related papers
- ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety DetectorsZhexin Zhang, Yida Lu, Jingyuan Ma, Di Zhang et al.EMNLP Findings 2024 · 14 citations
- SafeToolBench: Pioneering a Prospective Benchmark to Evaluating Tool Utilization Safety in LLMsHongfei Xia, Hongru Wang, Zeming Liu, Qian Yu et al.EMNLP Findings 2025 · 1 citation
- Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language ModelsYi Ding, Lijun Li, Bing Cao, Jing ShaoICLR 2026 · 21 citations
- ToolSafety: A Comprehensive Dataset for Enhancing Safety in LLM-Based Agent Tool InvocationsYuejin Xie, Youliang Yuan, Wenxuan Wang, Fan Mo et al.EMNLP 2025 · 7 citations
- Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow InstructionsFederico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger et al.ICLR 2024 · 373 citations
