Efficient Zero-Shot and Label-free Log Anomaly Detection for Resource-Constrained Systems

Logs, generated from modern computational systems such as cloud servers or DBMS, are the primary indicator of system states and thus have drawn significant attention from researchers. One of its key tasks is log anomaly detection, aiming to discover anomalous signals that can subsequently imply errors in systems. Conventionally, the major challenge of such a detection task is the insufficiency of well-labeled logs, which require unaffordable human resources. To tackle this issue, recent works have adopted the large language models (LLMs) as zeroshot label-free log anomaly detectors. However, these solutions necessitate direct deployment of LLM instances for downstream tasks, incurring substantial computational costs. Such costs limit their applications in resource-constrained scenarios, which motivates us to reposition the LLMs from direct detectors to training assistants and propose MaidLog. Specifically, MaidLog comprises an LLM-assisted pseudo-labels assignment workflow that automatically generates high-quality labels at training, enabling it to run entirely without manual labels. Based on them, we propose a lightweight detector designed for generalizability, where a well-trained detector is even applicable to downstream systems without any post-training. Once the training is finished, the downstream detection is efficient and only detector-based, without any involvement of LLM. Extensive experiments show that MaidLog effectively handles challenging real-world scenarios. With only a few unlabeled entries, MaidLog achieves comparable performance with a $25,000 \mathrm{x}$ speedup over SOTA LLM-centric solutions in out-of-domain systems, demonstrating its applicability on resource-constrained systems.