ACL Findings2026
LLMs Meet Isolation Kernel: Lightweight, Learning-free Binary Embeddings for Fast Retrieval
Zhibo Zhang, Yang Xu, Kai Ming Ting, Cam-Tu Nguyen
Abstract
Large language models (LLMs) have recently enabled remarkable progress in text representation. However, their embeddings are typically high-dimensional, leading to substantial storage and retrieval overhead. Although recent approaches such as Matryoshka Representation Learning (MRL) and Contrastive Sparse Representation (CSR) alleviate these issues to some extent, they still suffer from retrieval accuracy degradation. This paper proposes Isolation Kernel Embedding or IKE, a learning-free method that transforms an LLM embedding into a binary embedding using Isolation Kernel (IK). Lightweight and based on binary encoding, IKE offers a low memory footprint and fast bitwise computation, lowering retrieval latency. Experiments on multiple text retrieval datasets demonstrate that IKE offers up to 16.7× faster retrieval and 16× lower memory usage than the original LLM embeddings, while maintaining comparable accuracy. Theoretically, we show that IKE works because it satisfies four essential criteria for effective binary hashing that other methods do not possess. Compared to CSR, IKE consistently achieves better retrieval efficiency and effectiveness. IKE also works effectively with graph-based indexing, demonstrating its superiority in balancing accuracy and latency compared to alternative compression techniques in the approximate nearest neighbor (ANN) search setting.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on9
- Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation LearningWeixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung et al.NeurIPS 2022 · 834 citations
- Accelerating Large-Scale Inference with Anisotropic Vector QuantizationRuiqi Guo, Philip Sun, Erik Lindgren, Quan Geng et al.ICML 2020 · 539 citations
- Efficient Approximate Nearest Neighbor Search in Multi-dimensional DatabasesYun Peng, Byron Choi, Tsz Nam Chan, Jianye Yang et al.SIGMOD 2023 · 74 citations
- Starling: An I/O-Efficient Disk-Resident Graph Index Framework for High-Dimensional Vector Similarity Search on Data SegmentMengzhao Wang, Weizhi Xu, Xiaomeng Yi, Songlin Wu et al.SIGMOD 2024 · 63 citations
- Graph-Based Vector Search: An Experimental Evaluation of the State-of-the-ArtIlias Azizi, Karima Echihabi, Themis PalpanasSIGMOD 2025 · 36 citations
Related papers
- Text Anomaly Detection with Simplified Isolation KernelYang Cao, Sikun Yang, Yu-Jiu Yang, Lianyong Qi et al.EMNLP Findings 2025 · 2 citations
- Embedding Compression with Isotropic Iterative QuantizationSiyu Liao, Jie Chen, Yanzhi Wang, Qinru Qiu et al.AAAI 2020 · 16 citations
- Beyond Matryoshka: Revisiting Sparse Coding for Adaptive RepresentationTiansheng Wen, Yifei Wang, Zequn Zeng, Zhong Peng et al.ICML 2025
- CSRv2: Unlocking Ultra-Sparse EmbeddingsLixuan Guo, Yifei Wang, Tiansheng Wen, Yifan Wang et al.ICLR 2026 · 7 citations
- BTR: Binary Token Representations for Efficient Retrieval Augmented Language ModelsQingqing Cao, Sewon Min, Yizhong Wang, Hannaneh HajishirziICLR 2024 · 7 citations
