USENIX Security2026Top-tier venue
SoK: Attack and Defense Landscape of Agentic AI Systems
Juhee Kim, Wenbo Guo, Dawn Song
Abstract
AI agents that combine large language models with non-AI system components are rapidly emerging in real-world applications, offering unprecedented automation and flexibility. However, this unprecedented flexibility introduces complex security challenges that differ from those found in traditional software systems. This paper presents the first comprehensive systematization of knowledge on AI agent security, including an analysis of agents' design space, attack landscape, and defense mechanisms for secure AI agent systems. We further identify open challenges that point to promising directions for future research in this emerging domain. Our work introduces the first systematic framework for understanding the security risks and defense landscapes of AI agents, serving as a foundation for building both secure agentic systems and advancing research in this critical area.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext b7eb10c6-7002-4431-a50d-e0e1b81f9496Builds on38
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
- Gorilla: Large Language Model Connected with Massive APIsShishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. GonzalezNeurIPS 2024 · 1,715 citations
- AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge BasesZhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song et al.NeurIPS 2024 · 539 citations
Related papers
- Security Debt in LLM Agent Applications: A Measurement Study of Vulnerabilities and Mitigation Trade-offsZhuoxiang Shen, Jiarun Dai, Yuan Zhang, Min YangASE 2025 · 1 citation
- JARVIS or Ultron? A Survey on the Safety and Security Threats of Computer-Using AgentsAda Chen, Yongjiang Wu, Junyuan Zhang, Jingyu Xiao et al.ACL 2026 · 24 citations
- Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI AgentsJulia Bazinska, Max Mathys, Francesco Casucci, Mateo Rojas-Carulla et al.ICLR 2026 · 10 citations
- AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use AgentsHaitao Hu, Peng Chen, Yanpeng Zhao, Yuqi ChenCCS 2025
- A2ASecBench: A Protocol-Aware Security Benchmark for Agent-to-Agent Multi-Agent SystemsTianhao Li, Chuangxin Chu, Yujia Zheng, Bohan Zhang et al.ICLR 2026
