Challenges and opportunities: an in-depth empirical study on configuration error injection testing
Wang Li, Zhouyang Jia, Shanshan Li, Yuanliang Zhang, Teng Wang, Erci Xu, Ji Wang, Xiangke Liao
Abstract
Configuration error injection testing (CEIT) could systematically evaluate software reliability and diagnosability to runtime configuration errors. This paper explores the challenges and opportunities of applying CEIT technique. We build an extensible, highlymodularized CEIT framework named CeitInspector to experiment with various CEIT techniques. Using CeitInspector, we quantitatively measure the effectiveness and efficiency of CEIT using six mature and widely-used server applications. During this process, we find a fair number of test cases are left unstudied by the prior research work. The injected configuration errors in these cases often indicate latent misconfigurations, which might be ticking time bombs in the system and lead to severe damage. We conduct an in-depth study regarding these cases to reveal the root causes, and explore possible remedies. Finally, we come up with actionable suggestions guided by our study to improve the effectiveness and efficiency of the existing CEIT techniques.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
7 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- VUzzer: Application-aware Evolutionary Fuzzing cites Driller: Augmenting Fuzzing Through Selective Symbolic Execution
- ConfTainter: Static Taint Analysis For Configuration Options cites Directed Greybox Fuzzing
- ConfTainter: Static Taint Analysis For Configuration Options cites Understanding and Detecting On-The-Fly Configuration Bugs
- Large Language Models as Configuration Validators cites ConfTainter: Static Taint Analysis For Configuration Options
- Large Language Models as Configuration Validators cites Understanding and Detecting On-The-Fly Configuration Bugs
- Large Language Models as Configuration Validators cites Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via Logs
- Directed Greybox Fuzzing cites VUzzer: Application-aware Evolutionary Fuzzing
- Directed Greybox Fuzzing cites Driller: Augmenting Fuzzing Through Selective Symbolic Execution
- Understanding and Detecting On-The-Fly Configuration Bugs cites Directed Greybox Fuzzing
Cited by top-tier papers7
- Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via LogsShiwen Shan, Yintong Huo, Yuxin Su, Yichen Li et al.ISSTA 2024 · 18 citations
- ConfTainter: Static Taint Analysis For Configuration OptionsTeng Wang, Haochen He, Xiaodong Liu, Shanshan Li et al.ASE 2023 · 12 citations
- Understanding and Detecting On-The-Fly Configuration BugsTeng Wang, Zhouyang Jia, Shanshan Li, Si Zheng et al.ICSE 2023 · 12 citations
- Large Language Models as Configuration ValidatorsXinyu Lian, Yinfang Chen, Runxiang Cheng, Jie Huang et al.ICSE 2025 · 7 citations
- Unearthing Semantic Checks for Cloud Infrastructure-as-Code ProgramsYiming Qiu, Patrick Tser Jern Kon, Ryan Beckett, Ang ChenSOSP 2024 · 4 citations
Builds on3
- Driller: Augmenting Fuzzing Through Selective Symbolic ExecutionNick Stephens, John Grosen, Christopher Salls, Andrew Dutcher et al.NDSS 2016 · 1,021 citations
- Directed Greybox FuzzingMarcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, Abhik RoychoudhuryCCS 2017 · 836 citations
- VUzzer: Application-aware Evolutionary FuzzingSanjay Rawat, Vivek Jain, Ashish Kumar, Lucian Cojocar et al.NDSS 2017 · 700 citations
Related papers
- ECFuzz: Effective Configuration Fuzzing for Large-Scale SystemsJunqiang Li, Senyi Li, Keyao Li, Falin Luo et al.ICSE 2024 · 12 citations
- CAFault: Enhance Fault Injection Technique in Practical Distributed Systems via Abundant Fault-Dependent ConfigurationsYuanliang Chen, Fuchen Ma, Yuanhang Zhou, Zhen Yan et al.USENIX ATC 2025 · 7 citations
- Test-case prioritization for configuration testingRunxiang Cheng, Lingming Zhang, Darko Marinov, Tianyin XuISSTA 2021 · 34 citations
- CP-Detector: Using Configuration-related Performance Properties to Expose Performance BugsHaochen He, Zhouyang Jia, Shanshan Li, Erci Xu et al.ASE 2020 · 14 citations
- Fuzzing Error Handling Code using Context-Sensitive Software Fault InjectionZu-Ming Jiang, Jia-Ju Bai, Kangjie Lu, Shi-Min HuUSENIX Security 2020
