Dynamic Flat Filter: A Unified Framework for Scalable and Stable Fingerprint-Based Filters
Yang Du, Shankui Ji, He Huang, Yu-E Sun, Ge Gao, Guoju Gao
Abstract
Fingerprint-based filters, such as Cuckoo and Quotient Filters, are a key class of data structures for approximate membership query with deletion support. However, their fixed capacity is a major limitation in dynamic environments with unpredictable data volumes. While prior research has introduced dynamic filters, their resizing mechanisms are tightly coupled to specific filter structures, making their mechanisms difficult to generalize. Moreover, prior dynamic solutions introduce severe performance bottlenecks: chain-based or ring-based filters enable incremental growth yet degrade queries by probing multiple sub-filters, while global-resizing filters cause long, disruptive stalls during resizing as their single, large filter grows. In addition, prior incremental designs often leave scale-induced false positive rate growth unaddressed. In this paper, we present the Dynamic Flat Filter (DFF), a unified framework that decouples dynamic resizing from core filter logic, enabling fingerprint-based filters to achieve incremental scalability while preserving their original performance, memory efficiency, and false positive guarantees. DFF maintains a variable-sized set of independent segments, each reusing the base filter's native layout, enabling fine-grained resizing with only minor integration hooks. We achieve O(1) insertion, query, and deletion by employing a flat structure for all segments and a lookup table that quickly maps any item to its corresponding segment. Additionally, DFF incorporates a fingerprint growth strategy that keeps the false positive rate essentially constant as the filter scales. We demonstrate the general applicability of DFF by integrating it with Cuckoo and Quotient Filters, each requiring fewer than 60 lines of code to be modified, and evaluate these implementations on real-world and synthetic datasets. Experimental results show that DFF outperforms SOTA baselines in insertion, query, and deletion, maintains high memory efficiency, and stabilizes the false positive rate. Notably, DFF attains at least 1.33× the overall throughput on query-intensive workloads and reduces worst-case insertion latency by up to 59.5% compared with the best prior dynamic filter.
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Your agent calls
Lunesearch_papers
Free to start. No credit card required.
Terminal
Install the CLIlune papers get 8a5c2351-8e62-4d17-856b-0e844bc5ccb4Related papers
- InfiniFilter: Expanding Filters to Infinity and BeyondNiv Dayan, Ioana O. Bercea, Pedro Reviriego, Rasmus PaghSIGMOD 2023 · 27 citations
- Prefix Filter: Practically and Theoretically Better Than BloomTomer Even, Guy Even, Adam MorrisonVLDB 2022 · 13 citations
- Vacuum Filters: More Space-Efficient and Faster Replacement for Bloom and Cuckoo FiltersMinmei Wang, Mingxun Zhou, Shouqian Shi, Chen QianVLDB 2020 · 58 citations
- Aleph Filter: To Infinity in Constant TimeNiv Dayan, Ioana Oriana Bercea, Rasmus PaghVLDB 2024 · 13 citations
- The Logarithmic Dynamic Cuckoo FilterFan Zhang, Hanhua Chen, Hai Jin, Pedro ReviriegoICDE 2021 · 23 citations
