Peeling Close to the Orientability Threshold - Spatial Coupling in Hashing-Based Data Structures

In multiple-choice data structures each element x in a set S of m keys is associated with a random set e(x) ⊆ [n] of buckets with capacity ℓ ≥ 1 by hash functions. This setting is captured by the hypergraph H = ([n], e(x) | x ∈ S). Accomodating each key in an associated bucket amounts to finding an ℓ-orientation of H assigning to each hyperedge an incident vertex such that each vertex is assigned at most ℓ hyperedges. If each subhypergraph of H has minimum degree at most ℓ, then an ℓ-orientation can be found greedily and H is called ℓ-peelable. Peelability has a central role in invertible Bloom lookup tables and can speed up the construction of retrieval data structures, perfect hash functions and cuckoo hash tables.

Many hypergraphs exhibit sharp density thresholds with respect to ℓ-orientability and ℓ-peelability, i.e. as the density c = m n grows past a critical value, the probability of these properties drops from almost 1 to almost 0. In fully random k-uniform hypergraphs the thresholds c * k,ℓ for ℓ-orientability significantly exceed the thresholds for ℓ-peelability. In this paper, for every k ≥ 2 and ℓ ≥ 1 with (k, ℓ) = (2, 1) and every z > 0, we construct a new family of random k-uniform hypergraphs with i.i.d. random hyperedges such that both the ℓ-peelability and the ℓ-orientability thresholds approach c * k,ℓ as z → ∞. In particular we achieve 1-peelability at densities arbitrarily close to 1, extending the reach of greedy algorithms.

Our construction is simple: The n vertices are linearly ordered and each hyperedge selects its k elements uniformly at random from a random range of n z+1 consecutive vertices. We thus exploit the phenomenon of threshold saturation via spatial coupling discovered in the context of low-density parity-check codes. Once the connection to data structures is in plain sight, a framework by Kudekar, Richardson and Urbanke [39] does the heavy lifting in our proof.

We demonstrate the usefulness of our construction using our hypergraphs as a drop-in replacement in a retrieval data structure by Botelho et al. [8]. This reduces memory usage from ≈ 1.23m bits to ≈ 1.12m bits (for input size m). Using k > 3 attains, at small sacrifices in running time, further improvements to memory usage.