Sometimes Simpler is Better: A Comprehensive Analysis of State-of-the-Art Provenance-Based Intrusion Detection Systems

Provenance-based intrusion detection systems (PIDSs) have garnered significant attention from the research community over the past decade. Although recent studies report nearperfect detection performance, we show that these systems are not viable for practical deployment. We implemented eight state-of-the-art systems within a unified framework and identified nine key shortcomings that hinder their practical adoption. Through extensive experiments, we quantify the impact of these shortcomings using cybersecurity-oriented metrics and propose solutions to address them for real-world applicability. Building on these insights, we demonstrate that most existing systems add unnecessary complexity, whereas a simple neural network reaches state-of-the-art detection on eight of nine DARPA datasets while offering a lighter, faster, and real-time detection solution. Finally, we highlight critical open research challenges that remain unaddressed in the current literature, paving the way for future advancements. To support research, we open-source our framework and provide pre-processed datasets with ground truth to support consistent evaluation.