Lune

ACL Findings2026

Peering Behind the Shield: Guardrail Identification in Large Language Models

Ziqing Yang, Yixin Wu, Rui Wen, Michael Backes, Yang Zhang

Abstract

With the rapid adoption of large language models (LLMs), conversational AI agents have become widely deployed across real-world applications. To enhance safety, these agents are often equipped with guardrails that moderate harmful content. Identifying the guardrails in an agent thus becomes critical for adversaries to understand the system and design guardspecific attacks. In this work, we introduce AP-Test, a novel approach that leverages guardspecific adversarial prompts to detect the identity of guardrails deployed in black-box AI agents. Our method addresses key challenges in this task, including the influence of safetyaligned LLMs and other guardrails, as well as a lack of principled decision-making strategies. AP-Test employs two complementary testing strategies, input and output guard tests, and a new metric, match score, to enable robust identification. Experiments across diverse agents and four open-source guardrails demonstrate that AP-Test achieves perfect classification accuracy in multiple scenarios. Ablation studies further highlight the necessity of our proposed components. Our findings reveal a practical path toward guardrail identification in real-world AI systems. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Tree of Attacks: Jailbreaking Black-Box LLMs Automatically cites Multi-step Jailbreaking Privacy Attacks on ChatGPT
  • Tree of Attacks: Jailbreaking Black-Box LLMs Automatically cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
  • FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
  • AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models cites Multi-step Jailbreaking Privacy Attacks on ChatGPT
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on12

  1. Tree of Attacks: Jailbreaking Black-Box LLMs AutomaticallyAnay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson et al.NeurIPS 2024 · 835 citations
  2. AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language ModelsXiaogeng Liu, Nan Xu, Muhao Chen, Chaowei XiaoICLR 2024 · 722 citations
  3. FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual PromptsYichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang et al.AAAI 2025 · 350 citations
  4. Multi-step Jailbreaking Privacy Attacks on ChatGPTHaoran Li, Dadi Guo, Wei Fan, Mingshi Xu et al.EMNLP Findings 2023 · 204 citations
  5. InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model AgentsQiusi Zhan, Zhixiang Liang, Zifan Ying, Daniel KangACL Findings 2024 · 147 citations

Related papers

  1. LS-Guard: Adaptive Safety Guardrails Tailored to Individual LLMsJinggui Liang, Lizi LiaoACL Findings 2026
  2. ExpGuard: LLM Content Moderation in Specialized DomainsMinseok Choi, Dongjin Kim, Seungbin Yang, Subin Kim et al.ICLR 2026 · 3 citations
  3. On Guardrail Models' Robustness to Mutations and Adversarial AttacksElias Bassani, Ignacio SanchezEMNLP Findings 2025 · 2 citations
  4. AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety DetectionWeidi Luo, Shenghong Dai, Xiaogeng Liu, Suman Banerjee et al.ACL 2025 · 41 citations
  5. ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and FeedbackYutao Mou, Zhangchi Xue, Lijun Li, Peiyang Liu et al.ACL Findings 2026 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines