USENIX Security2026Top-tier venue
SoK: DARPA's AI Cyber Challenge (AIxCC): Competition Design, Architectures, and Lessons Learned
Cen Zhang, Younggi Park, Fabian Fleischer, Yu-Fu Fu, Jiho Kim, Dongkwan Kim, Youngjoon Kim, Qingxiao Xu, Andrew Chin, Ze Sheng, Hanqing Zhao, Michael Pelican
Abstract
DARPA's AI Cyber Challenge (AIxCC, 2023(AIxCC, -2025) ) is the largest competition to date for building fully autonomous Cyber Reasoning Systems (CRSs) that leverage recent advances in AI-particularly large language models (LLMs)-to discover and remediate vulnerabilities in real-world open-source software. This paper presents the first systematic analysis of AIxCC. Drawing on design documents, source code, execution traces, and discussions with organizers and all finalist teams, we examine the competition's structure and key design decisions, characterize the architectural approaches of finalist CRSs, and analyze competition results beyond the final scoreboard. Our analysis reveals the factors that truly drove CRS performance, identifies genuine technical advances achieved by teams, and exposes limitations that remain open for future research. We conclude with lessons for organizing future competitions and broader insights toward deploying autonomous CRSs in practice.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 6e1e2d25-3549-4202-8669-8d8db140c8adCited by top-tier papers2
- Contextualizing Sink Knowledge for Java Vulnerability DiscoveryFabian Fleischer, Cen Zhang, Joonun Jang, Jeongin Cho et al.S&P 2026 · 2 citations
- SnakeCharmer: Automatic Fuzzing Harness Generation for Pure and Hybrid Python LibrariesGabriel Sherman, Stefan NagyFSE 2026
Builds on11
- Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
- SWE-agent: Agent-Computer Interfaces Enable Automated Software EngineeringJohn Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret et al.NeurIPS 2024 · 2,059 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
- NAUTILUS: Fishing for Deep Bugs with GrammarsCornelius Aschermann, Tommaso Frassetto, Thorsten Holz, Patrick Jauernig et al.NDSS 2019 · 291 citations
- Ijon: Exploring Deep State Spaces via FuzzingCornelius Aschermann, Sergej Schumilo, Ali Abbasi, Thorsten HolzS&P 2020 · 146 citations
Related papers
- Rise of the HaCRS: Augmenting Autonomous Cyber Reasoning Systems with Human AssistanceYan Shoshitaishvili, Michael Weissbacher, Lukas Dresel, Christopher Salls et al.CCS 2017 · 57 citations
- Measuring and Augmenting Large Language Models for Solving Capture-the-Flag ChallengesZimo Ji, Daoyuan Wu, Wenyuan Jiang, Pingchuan Ma et al.CCS 2025
- PACEbench: A Framework for Evaluating Practical AI Cyber-Exploitation CapabilitiesZicheng Liu, Lige Huang, Jie Zhang, Dongrui Liu et al.ICLR 2026 · 6 citations
- Examining Zero-Shot Vulnerability Repair with Large Language ModelsHammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri et al.S&P 2023
- AICrypto: Evaluating Cryptography Capabilities of Large Language ModelsYu Wang, Yijian Liu, Liheng Ji, Han Luo et al.ICML 2026 · 3 citations
