Lune

ICLR2025Top-tier venue

R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning

Mintong Kang, Bo Li

Abstract

As large language models (LLMs) become increasingly prevalent across various applications, it is critical to establish safety guardrails to moderate input/output of LLMs and ensure compliance with safety policies. Existing guardrail models, such as OpenAI Mod and LlamaGuard, treat various safety categories (e.g., "self-harm", "self-harm/instructions") independently and fail to explicitly capture the intercorrelations among them. This has led to limitations such as ineffectiveness due to inadequate training on long-tail data from correlated safety categories, susceptibility to jailbreak attacks, and inflexibility regarding new safety categories. To address these limitations, we propose R 2 -Guard, a robust reasoning enabled LLM guardrail via knowledge-enhanced logical reasoning. Specifically, R 2 -Guard comprises two parts: the data-driven category-specific learning and reasoning components. The learning component provides unsafety probabilities of input on different safety categories. We then encode safety knowledge among different categories as first-order logical rules and embed them into a probabilistic graphic model (PGM) as the reasoning component. The unsafety probabilities of different categories from data-driven models are sent to the reasoning component for final inference. We employ two types of PGMs: Markov logic networks (MLNs) and probabilistic circuits (PCs), and optimize PCs to achieve precision-efficiency balance via improved graph structure. We also propose different methods to optimize the weights of knowledge. To further perform stress tests, we employ a pairwise construction method to construct a new safety benchmark TwinSafety, which features principled categories and presents new challenges for guardrail models. We show that R 2 -Guard is effective even given unrepresentative categories or challenging jailbreak prompts. We compare R 2 -Guard with eight strong guardrail models on six safety benchmarks, and demonstrate the robustness of R 2 -Guard against four SOTA jailbreak attacks. R 2 -Guard significantly surpasses LlamaGuard by 30.2% on ToxicChat and by 59.5% against jailbreak attacks. We further reveal that R 2 -Guard can effectively adapt to unseen safety categories by simply editing the reasoning graph.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

15 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Sok: Evaluating Jailbreak Guardrails for Large Language Models cites Training language models to follow instructions with human feedback
  • RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards cites Training language models to follow instructions with human feedback
  • Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
  • Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets cites Training language models to follow instructions with human feedback
  • Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Training language models to follow instructions with human feedback
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers15

  1. Reliable Weak-to-Strong Monitoring of LLM AgentsNeil Kale, Chen Bo Calvin Zhang, Kevin Zhu, Ankit Aich et al.ICLR 2026 · 28 citations
  2. Sok: Evaluating Jailbreak Guardrails for Large Language ModelsXunguang Wang, Zhenlan Ji, Wenxuan Wang, Zongjie Li et al.S&P 2026 · 27 citations
  3. Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning DatasetsLei Hsiung, Tianyu Pang, Yung-Chen Tang, Linyue Song et al.ACL 2026 · 22 citations
  4. RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguardsJingnan Zheng, Xiangtian Ji, Yijun Lu, Chenhang Cui et al.NeurIPS 2025 · 17 citations
  5. CogSys: Efficient and Scalable Neurosymbolic Cognition System via Algorithm-Hardware Co-DesignZishen Wan, Hanchen Yang, Ritik Raj, Che-Kai Liu et al.HPCA 2025 · 6 citations

Builds on16

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  4. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code GenerationJiawei Liu, Chunqiu Steven Xia, Yuyao Wang, Lingming ZhangNeurIPS 2023 · 2,317 citations
  5. Chatbot Arena: An Open Platform for Evaluating LLMs by Human PreferenceWei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos et al.ICML 2024 · 1,212 citations

Related papers

  1. MrGuard: A Multilingual Reasoning Guardrail for Universal LLM SafetyYahan Yang, Soham Dan, Shuo Li, Dan Roth et al.EMNLP 2025
  2. ThinkGuard: Deliberative Slow Thinking Leads to Cautious GuardrailsXiaofei Wen, Wenxuan Zhou, Wenjie Jacky Mo, Muhao ChenACL Findings 2025 · 2 citations
  3. LS-Guard: Adaptive Safety Guardrails Tailored to Individual LLMsJinggui Liang, Lizi LiaoACL Findings 2026
  4. Jailbreaking Large Language Models Against Moderation Guardrails via Cipher CharactersHaibo Jin, Andy Zhou, Joe D. Menke, Haohan WangNeurIPS 2024 · 55 citations
  5. Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM SafetyYuyou Zhang, Miao Li, William Han, Yihang Yao et al.ACL Findings 2025 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines