When Complex Event Recognition Meets Cloud-Native Architectures
Shizhe Liu, Haipeng Dai, Meng Li, Yuemeng Zhang, Shaoxu Song, Zhifeng Bao, Hancheng Wang, Xiaofeng Gao, Guihai Chen
Abstract
Complex Event Recognition (CER) aims to detect a predefined pattern composed of multiple primitive events. With the growing adoption of cloud-native techniques (i.e., computing and storage separation), which offer elasticity, availability, and cost efficiency, many database vendors are migrating their products to such architectures. However, when CER operates in cloud-native architectures, network becomes a performance bottleneck. To mitigate network-induced performance degradation, our key insight is to identify shorter time intervals that contain matches and transmit only the events within those intervals, hence reducing the transfer of irrelevant events. Upon this insight, we first propose a dual-filtering strategy that leverages both temporal and predicate constraints under multiple round-trips to incrementally shrink the time intervals. Then, we design two specialized filters: the shrinking window filter which reduces the complexity of time intervals maintenance from to , and the window-wise join filter, which enables low-cost round-trips for processing equality conditions. Furthermore, we propose a cost model to eliminate detrimental round-trips and prevent inefficiencies caused by excessively fine-grained round-trips. To the best of our knowledge, this is the first study to investigate CER in cloud-native architectures. Extensive evaluations demonstrate that our approach reduces transmission cost by over and achieves a to end-to-end query speedup across various evaluation engines (e.g., Flink and Esper) on the real-world and synthetic datasets, compared with the state-of-the-art approaches.
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Your agent calls
Lunesearch_papers
Free to start. No credit card required.
Terminal
Install the CLIlune papers get 7abf66c1-5f18-41b7-85a7-ebad9d8fc527Related papers
- ACER: Accelerating Complex Event Recognition via Two-Phase Filtering under Range Bitmap-Based IndexesShizhe Liu, Haipeng Dai, Shaoxu Song, Meng Li et al.KDD 2024 · 2 citations
- INEv: In-Network Evaluation for Event Stream ProcessingSamira Akili, Steven Purtzel, Matthias WeidlichSIGMOD 2023 · 13 citations
- CORE: a COmplex event Recognition EngineMarco Bucchi, Alejandro Grez, Andrés Quintana, Cristian Riveros et al.VLDB 2022 · 30 citations
- DecoPa: Query Decomposition for Parallel Complex Event ProcessingSamira Akili, Steven Purtzel, Matthias WeidlichSIGMOD 2024 · 8 citations
- FlexPushdownDB: Hybrid Pushdown and Caching in a Cloud DBMSYifei Yang, Matt Youill, Matthew E. Woicik, Yizhou Liu et al.VLDB 2021 · 67 citations
