Learning to Embed Categorical Features without Embedding Tables for Recommendation
Wang-Cheng Kang, Derek Zhiyuan Cheng, Tiansheng Yao, Xinyang Yi, Ting Chen, Lichan Hong, Ed H. Chi
Abstract
Embedding learning of categorical features (e.g. user/item IDs) is at the core of various recommendation models including matrix factorization and neural collaborative filtering. The standard approach creates an embedding table where each row represents a dedicated embedding vector for every unique feature value. However, this method fails to efficiently handle high-cardinality features and unseen feature values (e.g. new video ID) that are prevalent in real-world recommendation systems. In this paper, we propose an alternative embedding framework Deep Hash Embedding (DHE), replacing embedding tables by a deep embedding network to compute embeddings on the fly. DHE first encodes the feature value to a unique identifier vector with multiple hashing functions and transformations, and then applies a DNN to convert the identifier vector to an embedding. The encoding module is deterministic, non-learnable, and free of storage, while the embedding network is updated during the training time to learn embedding generation. Empirical results show that DHE achieves comparable AUC against the standard one-hot full embedding, with smaller model sizes. Our work sheds light on the design of DNN-based alternative embedding schemes for categorical features without using embedding table lookup.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
22 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Learnable Embedding sizes for Recommender Systems cites Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
- Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models cites Learnable Embedding sizes for Recommender Systems
- Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models cites Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
- Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models cites Recommender Systems with Generative Retrieval
- DreamShard: Generalizable Embedding Table Placement for Recommender Systems cites RecShard: statistical feature-based memory optimization for industry-scale neural recommendation
- DreamShard: Generalizable Embedding Table Placement for Recommender Systems cites Learnable Embedding sizes for Recommender Systems
- DreamShard: Generalizable Embedding Table Placement for Recommender Systems cites Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
Cited by top-tier papers22
- Recommender Systems with Generative RetrievalShashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan et al.NeurIPS 2023 · 474 citations
- Wukong: Towards a Scaling Law for Large-Scale RecommendationBuyun Zhang, Liang Luo, Yuxin Chen, Jade Nie et al.ICML 2024 · 108 citations
- RecShard: statistical feature-based memory optimization for industry-scale neural recommendationGeet Sethi, Bilge Acun, Niket Agarwal, Christos Kozyrakis et al.ASPLOS 2022 · 65 citations
- Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language ModelsXin Cheng, Wangding Zeng, Damai Dai, Qinyu Chen et al.ACL 2026 · 57 citations
- DreamShard: Generalizable Embedding Table Placement for Recommender SystemsDaochen Zha, Louis Feng, Qiaoyu Tan, Zirui Liu et al.NeurIPS 2022 · 37 citations
Builds on5
- ALBERT: A Lite BERT for Self-supervised Learning of Language RepresentationsZhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel et al.ICLR 2020 · 7,418 citations
- Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional DomainsMatthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil et al.NeurIPS 2020 · 4,036 citations
- Implicit Neural Representations with Periodic Activation FunctionsVincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell et al.NeurIPS 2020 · 4,008 citations
- Learnable Embedding sizes for Recommender SystemsSiyi Liu, Chen Gao, Yihong Chen, Depeng Jin et al.ICLR 2021 · 97 citations
- Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation SystemsHao-Jun Michael Shi, Dheevatsa Mudigere, Maxim Naumov, Jiyan YangKDD 2020 · 88 citations
Related papers
- GraphHash: Graph Clustering Enables Parameter Efficiency in Recommender SystemsXinyi Wu, Donald Loveland, Runjin Chen, Yozen Liu et al.WWW 2025 · 4 citations
- Hybrid Embedding Framework for Memory-Efficient Recommendation SystemsSeung Jin Yang, Hyuk-Jae Lee, Chae-Eun RheeDAC 2025
- Content-aware Neural Hashing for Cold-start RecommendationCasper Hansen, Christian Hansen, Jakob Grue Simonsen, Stephen Alstrup et al.SIGIR 2020 · 29 citations
- Probabilistic Hash Embeddings for Online Learning of Categorical FeaturesAodong Li, Abishek Sankararaman, Balakrishnan NarayanaswamyAAAI 2026
- Clustering the Sketch: Dynamic Compression for Embedding TablesHenry Ling-Hei Tsang, Thomas D. AhleNeurIPS 2023 · 5 citations
