StateTree: A Tree-Based Modeling Approach for Fault Detection in Recurrent Neural Networks

Recurrent Neural Networks (RNNs) have become a core component of modern intelligent software due to their strong ability to model temporal dependencies. As RNNs are increasingly deployed in safety-critical domains, ensuring their reliability is crucial. However, most existing testing techniques are designed for feedforward networks and struggle with RNNs. The stateful nature, recurrent feedback, and long-term dependencies of RNNs make it difficult for existing testing methods to capture decision logic and temporal behaviors, which in turn makes detecting fault-inducing behaviors that emerge through temporal decision evolution challenging. To address these challenges, we propose StateTree, a tree-based abstract modeling approach for systematic testing of RNN-based systems. StateTree constructs an Abstract State Tree (AST) that captures major RNN decision behaviors, where each root-to-leaf path represents an abstract decision for intuitive interpretation and structured exploration. Using the AST, StateTree guides the testing process toward both major and previously unseen paths to reveal erroneous behaviors. Experiments show that StateTree accurately abstracts RNN decisions, detects hundreds of faults, and retraining with its identified test cases improves robustness beyond existing RNN coverage methods, demonstrating its effectiveness in both fault detection and model performance enhancement.