Quantifying Frontier LLM Capabilities for Container Sandbox Escape

Large language models (LLMs) increasingly act as autonomous agents, using tools to execute code, read and write files, and access networks, creating novel security risks. To mitigate these risks, agents are commonly deployed and evaluated in isolated "sandbox" environments, often implemented using Docker/OCI containers. We introduce SANDBOXESCAPEBENCH, an open benchmark that safely measures an LLM's capacity to break out of these sandboxes. The benchmark is implemented as an inspect-ai Capture the Flag (CTF) evaluation utilising a nested sandbox architecture with the outer layer containing the flag and no known vulnerabilities. Following a threat model of a motivated adversarial agent with shell access inside a container, SANDBOXESCAPEBENCH covers a spectrum of sandbox-escape mechanisms spanning misconfiguration, privilege allocation mistakes, kernel flaws, and runtime/orchestration weaknesses. We find that, when vulnerabilities are added, LLMs are able to identify and exploit them. This demonstrates the need for evaluations like SANDBOX-ESCAPEBENCH to ensure that sandboxing continues to provide the encapsulation needed for highly capable models. 1 1 Within 2 months of publication, several additional models were released: GPT 5.4-Pro and 5.5, Claude Opus 4.6 and Mythos Preview (using the same Mythos Preview checkpoint used to generate the results in this UK AISI blog post). We found that Claude Mythos Preview saturates this public benchmark, so deemed it important to update the paper. We report these revised findings in Section 6 along with longer-horizon (100Mtoken) runs. Updates in orange * Equal contribution