NebulaStream: An Adaptive and Efficient Multi-Query Stream Processing Engine

Existing Stream Processing Engines (SPEs), designed for scale-out in the cloud, often overlook the need for scaleup performance on heterogeneous hardware. As a result, they provide latency and throughput guarantees at the expense of high overhead on system resources, for example, memory and CPU usage. This over-provisioning is tolerable in cloud environments where resources are generously available. However, it is detrimental for low-end devices that are becoming increasingly common in today's cloud-edge environments. In this paper, we present the architecture of our adaptive and efficient multi-query stream processing engine, which is the core of the open-source system NebulaStream. Our engine is optimized for resource-constrained devices and their special requirements. In particular, our engine achieves better hardware utilization due to several key engine design decisions: 1) a task-based execution model for multi-query, scale-up execution based on an order-aware data flow model, 2) work-stealing-aware operators using hardware-tailored code generation, 3) runtime compute caching, and 4) work-stealing-aware state management. As a result, our novel engine outperforms general-purpose systems by at least one order of magnitude, executes hundreds of queries in parallel without performance degradation, and seamlessly reacts to fluctuating ingestion rates.