Interpretable Federated Transformer Log Learning for Cloud Threat Forensics
Gonzalo De La Torre Parra, Luis Selvera, Joseph Khoury, Hector Irizarry, Elias Bou-Harb, Paul Rad
Abstract
—Threat detection and forensics have become an imperative objective for any digital forensic triage. Supervised approaches have been proposed for inferring system and network anomalies; including anomaly detection contributions using syslogs. Nevertheless, most works downplay the importance of the interpretability of a model’s decision-making process. In this research, we are among the first to propose an interpretable federated transformer log learning model for threat detection supporting explainable cyber forensics. The proposed model is generated by training a local transformer-based threat detection model at each client in an organizational unit. Local models learn the system’s normal behavior from the syslogs which keep records of execution flows. Subsequently, a federated learning server aggregates the learned model parameters from local models to generate a global federated learning model. Log time-series capturing normal behavior are expected to differ from those possessing cyber threat activity. We demonstrate this difference through a goodness of fit test based on Karl-Pearson’s Chi-square statistic. To provide insights on actions triggering this difference, we integrate an attention-based interpretability module. We implement and evaluate our proposed model using HDFS, a publicly available log dataset, and an in-house collected and publicly-released dataset named CTDD, which consists of more than 8 million syslogs representing cloud collaboration services and systems compromised by different classes of cyber threats. Moreover, through different experiments, we demonstrate the log agnostic capability and applicability of our approach on real-world operational settings such as edge computing systems. Our interpretability module manifests significant attention difference between normal and abnormal logs which provide insightful interpretability of the model’s decision-making process. Finally, we deem the obtained results as a validation for the appropriate adoption of our approach in achieving threat forensics in the real world.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- "Get in Researchers; We're Measuring Reproducibility": A Reproducibility Study of Machine Learning Papers in Tier 1 Security Conferences cites DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
- SoK: History is a Vast Early Warning System: Auditing the Provenance of System Intrusions cites Towards a Timely Causality Analysis for Enterprise Security
- SoK: History is a Vast Early Warning System: Auditing the Provenance of System Intrusions cites DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
- Entente: Cross-silo Intrusion Detection on Network Log Graphs with Federated Learning cites Towards a Timely Causality Analysis for Enterprise Security
- Entente: Cross-silo Intrusion Detection on Network Log Graphs with Federated Learning cites SoK: History is a Vast Early Warning System: Auditing the Provenance of System Intrusions
Cited by top-tier papers3
- "Get in Researchers; We're Measuring Reproducibility": A Reproducibility Study of Machine Learning Papers in Tier 1 Security ConferencesDaniel Olszewski, Allison Lu, Carson Stillman, Kevin Warren et al.CCS 2023 · 19 citations
- Entente: Cross-silo Intrusion Detection on Network Log Graphs with Federated LearningJiacen Xu, Chenang Li, Yu Zheng, Zhou LiNDSS 2026 · 3 citations
- SoK: History is a Vast Early Warning System: Auditing the Provenance of System IntrusionsMuhammad Adil Inam, Yinfang Chen, Akul Goyal, Jason Liu et al.S&P 2023
Builds on3
- DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep LearningMin Du, Feifei Li, Guineng Zheng, Vivek SrikumarCCS 2017 · 1,823 citations
- Towards a Timely Causality Analysis for Enterprise SecurityYushan Liu, Mu Zhang, Ding Li, Kangkook Jee et al.NDSS 2018 · 177 citations
- UNVEIL: A Large-Scale, Automated Approach to Detecting RansomwareAmin Kharraz, Sajjad Arshad, Collin Mulliner, William K. Robertson et al.USENIX Security 2016
Related papers
- Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal AttentionCheryl Lee, Tianyi Yang, Zhuangbin Chen, Yuxin Su et al.ICSE 2023 · 52 citations
- MetaLog: Generalizable Cross-System Anomaly Detection from Logs with Meta-LearningChenyangguang Zhang, Tong Jia, Guopeng Shen, Pinyan Zhu et al.ICSE 2024 · 28 citations
- Log-based Anomaly Detection with Deep Learning: How Far Are We?Van-Hoang Le, Hongyu ZhangICSE 2022 · 212 citations
- Enabling Efficient Cyber Threat Hunting With Cyber Threat IntelligencePeng Gao, Fei Shao, Xiaoyuan Liu, Xusheng Xiao et al.ICDE 2021 · 124 citations
- LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly DetectionHongcheng Guo, Jian Yang, Jiaheng Liu, Jiaqi Bai et al.AAAI 2024 · 68 citations
