ACL Findings2026
Concept Tokens: Learning Behavioral Embeddings Through Concept Definitions
Abstract
We propose Concept Tokens, a lightweight method that adds a new special token to a pretrained LLM and learns only its embedding from multiple natural language definitions of a target concept, where occurrences of the concept are replaced by the new token. The LLM is kept frozen and the embedding is optimized with the standard language-modeling objective. We evaluate Concept Tokens in three settings. First, we study hallucinations in closedbook question answering on HotpotQA and find a directional effect: negating the hallucination token reduces hallucinated answers mainly by increasing abstentions, whereas asserting it increases hallucinations and lowers precision. Second, we induce recasting, a pedagogical feedback strategy for second language teaching, and observe the same directional effect. Moreover, compared to providing the full definitional corpus in-context, concept tokens better preserve compliance with other instructions (e.g., asking follow-up questions). Finally, we include a qualitative study with the Eiffel Tower and a fictional "Austral Tower" to illustrate what information the learned embeddings capture and where their limitations emerge. Overall, Concept Tokens provide a compact control signal learned from definitions that can steer behavior in frozen LLMs. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Extracting Latent Steering Vectors from Pretrained Language Models cites The Power of Scale for Parameter-Efficient Prompt Tuning
- 500xCompressor: Generalized Prompt Compression for Large Language Models cites The Power of Scale for Parameter-Efficient Prompt Tuning
Builds on7
- The Power of Scale for Parameter-Efficient Prompt TuningBrian Lester, Rami Al-Rfou, Noah ConstantEMNLP 2021 · 94 citations
- 500xCompressor: Generalized Prompt Compression for Large Language ModelsZongqian Li, Yixuan Su, Nigel CollierACL 2025 · 35 citations
- Extracting Latent Steering Vectors from Pretrained Language ModelsNishant Subramani, Nivedita Suresh, Matthew E. PetersACL Findings 2022 · 32 citations
- Controlling Large Language Models Through Concept Activation VectorsHanyu Zhang, Xiting Wang, Chengao Li, Xiang Ao et al.AAAI 2025 · 26 citations
- Toward Efficient Sparse Autoencoder-Guided Steering for Improved In-Context Learning in Large Language ModelsIkhyun Cho, Julia HockenmaierEMNLP 2025 · 3 citations
Related papers
- Alleviating Hallucinations from Knowledge Misalignment in Large Language Models via Selective Abstention LearningLei Huang, Xiaocheng Feng, Weitao Ma, Yuchun Fan et al.ACL 2025
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language ModelsYung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim et al.ICLR 2024 · 354 citations
- SinkTrack: Attention Sink based Context Anchoring for Large Language ModelsXu Liu, Guikun Chen, Wenguan WangICLR 2026 · 4 citations
- SH2: Self-Highlighted Hesitation Helps You Decode More TruthfullyJushi Kai, Tianhang Zhang, Hai Hu, Zhouhan LinEMNLP Findings 2024 · 5 citations
- Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired DecodingWeixu Zhang, Fanghua Ye, Qiang Gao, Jian Li et al.ACL Findings 2026 · 1 citation
