EMNLP Findings2025
Length Representations in Large Language Models
Sangjun Moon, Dasom Choi, Jingun Kwon, Hidetaka Kamigaito, Manabu Okumura
Abstract
Large language models (LLMs) have shown remarkable capabilities across various tasks, that are learned from massive amounts of text-based data. Although LLMs can control output sequence length, particularly in instruction-based settings, the internal mechanisms behind this control have been unexplored yet. In this study, we provide empirical evidence on how output sequence length information is encoded within the internal representations in LLMs. In particular, our findings show that multi-head attention mechanisms are critical in determining output sequence length, which can be adjusted in a disentangled manner. By scaling specific hidden units within the model, we can control the output sequence length without losing the informativeness of the generated text, thereby indicating that length information is partially disentangled from semantic information. Moreover, some hidden units become increasingly active as prompts become more length-specific, thus reflecting the model's internal awareness of this attribute. Our findings suggest that LLMs have learned robust and adaptable internal mechanisms for controlling output length without any external control.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- QLoRA: Efficient Finetuning of Quantized LLMs cites Finetuned Language Models are Zero-Shot Learners
- QLoRA: Efficient Finetuning of Quantized LLMs cites Training language models to follow instructions with human feedback
- Large Language Models are Zero-Shot Reasoners cites Language Models are Few-Shot Learners
- Large Language Models are Zero-Shot Reasoners cites Training language models to follow instructions with human feedback
- Finetuned Language Models are Zero-Shot Learners cites Language Models are Few-Shot Learners
- Finetuned Language Models are Zero-Shot Learners cites Training language models to follow instructions with human feedback
- Training language models to follow instructions with human feedback cites Finetuned Language Models are Zero-Shot Learners
- Training language models to follow instructions with human feedback cites Language Models are Few-Shot Learners
Cited by top-tier papers1
Builds on22
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
- Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
Related papers
- Emergent Response Planning in LLMsZhichen Dong, Zhanhui Zhou, Zhixuan Liu, Chao Yang et al.ICML 2025
- Hansel: Output Length Controlling Framework for Large Language ModelsSeoha Song, Junhyun Lee, Hyeonmok KoAAAI 2025 · 2 citations
- Computation Mechanism Behind LLM Position GeneralizationChi Han, Heng JiACL 2025
- LLM Circuit Analyses Are Consistent Across Training and ScaleCurt Tigges, Michael Hanna, Qinan Yu, Stella BidermanNeurIPS 2024 · 71 citations
- Do LLMs "know" internally when they follow instructions?Juyeon Heo, Christina Heinze-Deml, Oussama Elachqar, Kwan Ho Ryan Chan et al.ICLR 2025
