ConfLogger: Enhance Systems' Configuration Diagnosability through Configuration Logging
Shiwen Shan, Yintong Huo, Yuxin Su, Zhining Wang, Dan Li, Zibin Zheng
Abstract
Modern configurable systems, including distributed systems and recently popular decentralized systems in Web 3.0, offer customization via intricate configuration spaces, yet such flexibility introduces pervasive configuration-related issues such as misconfigurations and latent software bugs. Existing diagnosability supports focus on post-failure analysis of software behavior to identify configuration issues, but none of these approaches look into whether the software clues sufficient failure information for diagnosis. To fill in the blank, we propose the idea of configuration logging to enhance existing logging practices at the source code level. We develop ConfLogger, the first tool that unifies configuration-aware static taint analysis with LLM-based log generation to enhance software configuration diagnosability. Specifically, our method 1) identifies configuration-sensitive code segments by tracing configuration-related data flow in the whole project, and 2) generates diagnostic logging statements by analyzing configuration code contexts. Evaluation results on eight popular software systems demonstrate the effectiveness of ConfLogger to enhance configuration diagnosability. Specifically, ConfLogger-enhanced logs successfully aid a log-based misconfiguration diagnosis tool to achieve 100% accuracy on error localization in 30 silent misconfiguration scenarios, with 80% directly resolvable through explicit configuration information exposed. In addition, ConfLogger achieves 74% coverage of existing logging points, outperforming baseline LLM-based loggers by 12% and 30%. It also gains 8.6% higher in precision, 79.3% higher in recall, and 26.2% higher in F1 compared to the state-of-the-art baseline in terms of variable logging while also augmenting diagnostic value. A controlled user study on 22 cases further validated its utility, speeding up diagnostic time by 1.25× and improving troubleshooting accuracy by 251.4%.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on19
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep LearningMin Du, Feifei Li, Guineng Zheng, Vivek SrikumarCCS 2017 · 1,823 citations
- LILAC: Log Parsing using LLMs with Adaptive Parsing CacheZhihan Jiang, Jinyang Liu, Zhuangbin Chen, Yichen Li et al.FSE 2024 · 85 citations
- Testing Configuration Changes in Context to Prevent Production FailuresXudong Sun, Runxiang Cheng, Jianyan Chen, Elaine Ang et al.OSDI 2020 · 61 citations
- Using Deep Learning to Generate Complete Log StatementsAntonio Mastropaolo, Luca Pascarella, Gabriele BavotaICSE 2022 · 60 citations
Related papers
- Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via LogsShiwen Shan, Yintong Huo, Yuxin Su, Yichen Li et al.ISSTA 2024 · 18 citations
- ConfTainter: Static Taint Analysis For Configuration OptionsTeng Wang, Haochen He, Xiaodong Liu, Shanshan Li et al.ASE 2023 · 12 citations
- DiagConfig: Configuration Diagnosis of Performance Violations in Configurable Software SystemsZhiming Chen, Pengfei Chen, Peipei Wang, Guangba Yu et al.FSE 2023 · 9 citations
- UniLog: Automatic Logging via LLM and In-Context LearningJunjielong Xu, Ziang Cui, Yuan Zhao, Xu Zhang et al.ICSE 2024 · 56 citations
- Go Static: Contextualized Logging Statement GenerationYichen Li, Yintong Huo, Renyi Zhong, Zhihan Jiang et al.FSE 2024 · 17 citations
