LightToken: A Task and Model-agnostic Lightweight Token Embedding Framework for Pre-trained Language Models
Haoyu Wang, Ruirui Li, Haoming Jiang, Zhengyang Wang, Xianfeng Tang, Bin Bi, Monica Xiao Cheng, Bing Yin, Yaqing Wang, Tuo Zhao, Jing Gao
2023Year
5Citations
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Your agent calls
Lunesearch_papers
Free to start. No credit card required.
Terminal
Install the CLIlune papers get 6c092c49-7d19-4729-9a2c-15d0f6ff8e77Related papers
- Token Distillation: Attention-Aware Input Embeddings for New TokensKonstantin Dobler, Desmond Elliott, Gerard de MeloICLR 2026 · 7 citations
- Zero-Shot Tokenizer TransferBenjamin Minixhofer, Edoardo Maria Ponti, Ivan VulicNeurIPS 2024 · 37 citations
- MPCFORMER: Fast, Performant and Provate Transformer Inference with MPCDacheng Li, Hongyi Wang, Rulin Shao, Han Guo et al.ICLR 2023
- Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language ModelsTyler A. Chang, Yifan Xu, Weijian Xu, Zhuowen TuACL 2021
- Context-level Language Modeling by Learning Predictive Context Embeddingsbeiya dai, Yuliang Liu, Yunchong Song, Daozheng Xue et al.ICML 2026 · 5 citations
