Lune

EuroSys2026Top-tier venue

CSnake: Detecting Self-Sustaining Cascading Failure via Causal Stitching of Fault Propagations

Shangshu Qian, Lin Tan, Yongle Zhang

Abstract

Recent studies have revealed that self-sustaining cascading failures in distributed systems frequently lead to widespread outages, which are challenging to contain and recover from. Existing failure detection techniques struggle to expose such failures prior to deployment, as they typically require a complex combination of specific conditions to be triggered. This challenge stems from the inherent nature of cascading failures, as they typically involve a sequence of fault propagations, each activated by distinct conditions.

This paper presents CSnake, a fault injection framework to expose self-sustaining cascading failures in distributed systems. CSnake uses the novel idea of causal stitching, which causally links multiple single-fault injections in different test workloads to simulate complex fault propagation chains. To identify propagation chains between faults, CSnake designs a counterfactual causality analysis of fault propagationsfault causality analysis (FCA): FCA compares the execution trace of a fault injection run with its corresponding profile run (i.e., running the same test without the injection) and identifies any additional faults triggered, which are considered to have a causal relationship with the injected fault.

To address the large search space of fault and workload combinations, CSnake employs a three-phase allocation (3PA) protocol of test budget that prioritizes faults with unique and diverse causal consequences, thereby increasing the likelihood of uncovering conditional fault propagations. Furthermore, to avoid incorrectly connecting fault propagations from workloads with incompatible conditions, CSnake performs a local compatibility check that approximately checks the compatibility of the path constraints associated with connected fault propagations with low overhead.

CSnake has detected 15 bugs that resulted in self-sustaining cascading failures in five widely deployed distributed systems, five of which have been confirmed with two fixed.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • HEALER: Relation Learning Guided Kernel Fuzzing cites Directed Greybox Fuzzing
  • RetroWrite: Statically Instrumenting COTS Binaries for Fuzzing and Sanitization cites Directed Greybox Fuzzing
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on26

  1. Directed Greybox FuzzingMarcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, Abhik RoychoudhuryCCS 2017 · 836 citations
  2. RetroWrite: Statically Instrumenting COTS Binaries for Fuzzing and SanitizationSushant Dinesh, Nathan Burow, Dongyan Xu, Mathias PayerS&P 2020 · 187 citations
  3. HEALER: Relation Learning Guided Kernel FuzzingHao Sun, Yuheng Shen, Cong Wang, Jianzhong Liu et al.SOSP 2021 · 59 citations
  4. TCP-Fuzz: Detecting Memory and Semantic Bugs in TCP Stacks with FuzzingYonghao Zou, Jia-Ju Bai, Jielong Zhou, Jianfeng Tan et al.USENIX ATC 2021 · 53 citations
  5. Automatic Reliability Testing For Cluster Management ControllersXudong Sun, Wenqing Luo, Jiawei Tyler Gu, Aishwarya Ganesan et al.OSDI 2022 · 44 citations

Related papers

  1. CAFault: Enhance Fault Injection Technique in Practical Distributed Systems via Abundant Fault-Dependent ConfigurationsYuanliang Chen, Fuchen Ma, Yuanhang Zhou, Zhen Yan et al.USENIX ATC 2025 · 7 citations
  2. Efficient Exposure of Partial Failure Bugs in Distributed Systems with Inferred Abstract StatesHaoze Wu, Jia Pan, Peng HuangNSDI 2024 · 15 citations
  3. Efficient Reproduction of Fault-Induced Failures in Distributed Systems with Feedback-Driven Fault InjectionJia Pan, Haoze Wu, Tanakorn Leesatapornwongsa, Suman Nath et al.SOSP 2024 · 4 citations
  4. KernelRCA: Facilitating Root Cause Analysis of Memory Corruptions in Linux Kernel with Contextual Causality ChainKangzheng Gu, Yifan Zhang, Yuan Zhang, Min YangUSENIX Security 2026
  5. Rose: Reproducing External-Fault-Induced Failures in Distributed Systems with Lightweight InstrumentationSebastião Amaro, Pedro Fonseca, Miguel MatosEuroSys 2026
Dusk over the sea between two cliffs drawn in fine vertical lines