Lune

ACL Findings2024

UNIWIZ: A Unified Large Language Model Orchestrated Wizard for Safe Knowledge Grounded Conversations

Souvik Das, Rohini K. Srihari

Abstract

Warning: This paper contains disturbing language.

Large Language Models (LLMs) have made significant progress in integrating safety and knowledge alignment. However, adversarial actors can manipulate these models into generating unsafe responses, and excessive safety alignment can lead to unintended hallucinations. To address these challenges, we introduce UNIWIZ, a novel 2-step data orchestration framework that unifies safety and knowledge data generation. We propose a "safetypriming" method to generate synthetic safety data and overcome safety bottlenecks. We also inject relevant knowledge into conversations by retrieving factual information from curated sources. UNIWIZ dataset consists of 17, 638 quality-controlled conversations and 10, 000 augmented preference data. Pretrained models fine-tuned on UNIWIZ show improvements across various metrics and outperform stateof-the-art instruction-tuned models trained on much larger datasets.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • QLoRA: Efficient Finetuning of Quantized LLMs cites LoRA: Low-Rank Adaptation of Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on20

  1. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  2. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  3. QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
  4. TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
  5. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionAkari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil et al.ICLR 2024 · 1,798 citations

Related papers

  1. Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language ModelsYongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang et al.ICML 2024 · 140 citations
  2. StealthGraph: Exposing Domain-Specific Risks in LLMs through Knowledge-Graph-Guided Harmful Prompt GenerationHuawei Zheng, Xinqi Jiang, Sen Yang, Shouling Ji et al.ACL 2026 · 1 citation
  3. Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?Yanbo Wang, Jiyang Guan, Jian Liang, Ran HeCVPR 2025
  4. What Makes and Breaks Safety Fine-tuning? A Mechanistic StudySamyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy et al.NeurIPS 2024 · 62 citations
  5. Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language ModelsFei Wang, Ninareh Mehrabi, Palash Goyal, Rahul Gupta et al.EMNLP 2024 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines