Prompt Yourself: Awakening Textual Semantics in 1D Visual Tokenizers
Hualiang Wang, Siming Fu, Weinan Jia, Yuning Lu, Mu Liu, Jidong Jiang, Xiaomeng Li
Abstract
One-dimensional (1D) visual tokenizers offer notable semantic compactness by discarding local spatial priors, and have become increasingly popular for image reconstruction and generation tasks. However, such global and sequential representations struggle to preserve fine-grained visual content; simply increasing network size or token count offers only superficial mitigation. To address this, we introduce VLTok, a novel 1D hybrid tokenizer that unifies Visual and Language representations in a shared Token space through a self-prompted training paradigm. During training, VLTok simultaneously generates 1D visual and textual tokens from images, aligning the textual tokens with embeddings from a pre-trained language model. This crossmodal alignment infuses implicit linguistic cues into the tokenizer, enhancing fine-grained image encoding. During inference, the self-prompted paradigm eliminates the need for external text, maintaining the simplicity of the image-only framework while benefiting from multi-modal guidance. Extensive experiments on the ImageNet benchmark demonstrate that VLTok achieves state-of-the-art performance in both image reconstruction and image generation. For example, under the same model parameter budget, our method achieves a relative reduction of 11.1% in rFID and 18.7% in gFID compared to GigaTok.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 522cb502-ae40-4648-9c8e-a0d90be3e87cBuilds on32
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 citations
Related papers
- Concept-Guided Tokenization: Closing the Gap Between Reconstruction and GenerationYunqiao Yang, Haokun Lin, Guanzhong Wu, Ying WeiICML 2026
- GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image GenerationTianwei Xiong, Jun Hao Liew, Zilong Huang, Jiashi Feng et al.ICCV 2025 · 2 citations
- End-to-End Autoregressive Image Generation with 1D Semantic TokenizerWenda Chu, Bingliang Zhang, Jiaqi Han, Yizhuo Li et al.ICML 2026 · 2 citations
- Vision Foundation Models as Effective Visual Tokenizers for Autoregressive GenerationAnlin Zheng, Xin Wen, Xuanyang Zhang, Chuofan Ma et al.NeurIPS 2025 · 21 citations
- Adapting Self-Supervised Representations as a Latent Space for Efficient GenerationMing Gui, Johannes Schusterbauer, Timy Phan, Felix Krause et al.ICLR 2026 · 13 citations
