MQH: Locality Sensitive Hashing on Multi-level Quantization Errors for Point-to-Hyperplane Distances
Kejing Lu, Yoshiharu Ishikawa, Chuan Xiao
Abstract
Point-to-hyperplane nearest neighbor search (P2HNNS) is a fundamental problem which has many applications in data mining and machine learning. In this paper, we propose a provable Locality-Sensitive-Hashing (LSH) scheme based on multi-level quantization errors to solve this problem. In the indexing phase, for each data point, we compute the hash values of its residual vectors generated by a stepwise quantization process. In the query phase, for each processed point, we first determine its suitable level for hashing and then determine the size of hash bucket based on its quantization error in that level. We theoretically show that this treatment not only yields a probability guarantee on query results, but also makes the generated hash functions much more efficient to prune those false points. Experimental results on five real datasets show that the proposed approach generally runs 2X-10X faster than the state-of-the-art LSH-based approaches.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 3966bbce-1c44-47f7-b1ae-e7431b5d9ed4Builds on6
- Accelerating Large-Scale Inference with Anisotropic Vector QuantizationRuiqi Guo, Philip Sun, Erik Lindgren, Quan Geng et al.ICML 2020 · 539 citations
- HVS: Hierarchical Graph Structure Based on Voronoi Diagrams for Solving Approximate Nearest Neighbor SearchKejing Lu, Mineichi Kudo, Chuan Xiao, Yoshiharu IshikawaVLDB 2022 · 70 citations
- R2LSH: A Nearest Neighbor Search Scheme Based on Two-dimensional Projected SpacesKejing Lu, Mineichi KudoICDE 2020 · 40 citations
- Norm-Explicit Quantization: Improving Vector Quantization for Maximum Inner Product SearchXinyan Dai, Xiao Yan, Kelvin Kai Wing Ng, Jiu Liu et al.AAAI 2020 · 34 citations
- Point-to-Hyperplane Nearest Neighbor Search Beyond the Unit HypersphereQiang Huang, Yifan Lei, Anthony K. H. TungSIGMOD 2021 · 17 citations
Related papers
- PM-LSH: A Fast and Accurate LSH Framework for High-Dimensional Approximate NN SearchBolong Zheng, Xi Zhao, Lianggui Weng, Nguyen Quoc Viet Hung et al.VLDB 2020 · 64 citations
- VHP: Approximate Nearest Neighbor Search via Virtual Hypersphere PartitioningKejing Lu, Hongya Wang, Wei Wang, Mineichi KudoVLDB 2020 · 9 citations
- DB-LSH: Locality-Sensitive Hashing with Query-based Dynamic BucketingYao Tian, Xi Zhao, Xiaofang ZhouICDE 2022 · 21 citations
- MP-RW-LSH: An Efficient Multi-Probe LSH Solution to ANNS-L_1Huayi Wang, Jingfan Meng, Long Gong, Jun Xu et al.VLDB 2021 · 3 citations
- DET-LSH: A Locality-Sensitive Hashing Scheme with Dynamic Encoding Tree for Approximate Nearest Neighbor SearchJiuqi Wei, Botao Peng, Xiaodong Lee, Themis PalpanasVLDB 2024 · 35 citations
