PoCE: Automated Proof-of-Concept Synthesis using Large Language Models for Robust Validation

Vulnerability reports play a critical role in software repair, with Proof-of-Concept (PoC) tests serving as one of their most essential components. PoC tests enable software developers to reliably reproduce reported vulnerabilities and subsequently deploy patches. However, generating effective PoCs is costly, expertise-intensive, and increasingly challenging due to the diversity of modern software ecosystems and their complex dependencies. Inadequate or incorrect PoCs can significantly delay patch deployment, thereby increasing the window of exposure to attacks. Prior work on automated PoC generation struggles to produce comprehensive and reliable testing. In this work, we present an automated PoC generation framework, PoCE, capable of generating PoCs across diverse software systems by handling varied input formats and complex execution contexts using large language models. PoCE integrates structured in-context learning, retrieval-augmented generation, and iterative chain-of-thought reasoning to expand an initial successful PoC into multiple validated variants. These variants are executed in controlled environments to confirm success. We evaluate PoCE on thirteen widely used software projects, including TensorFlow, Yasm, Zlib, Liblouis, Cflow, Pytorch, Node.js, TCPDUMP, Fig2dev, Binutils, libsndfile, LibTIFF, and libsixel. Our approach achieves a success rate of 77.7% and generates multiple PoC variants for the most vulnerable cases, uncovering alternative trigger paths and edge conditions. We discover 68 zero-day PoCs and identify 26 previously unknown zero-day vulnerabilities in cross-layer software.