Lune

EMNLP Findings2024

PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness

Noah Wang, Feiyu Duan, Yibo Zhang, Wangchunshu Zhou, Ke Xu, Wenhao Huang, Jie Fu

Abstract

Large Language Models (LLMs) demonstrate impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding. Despite these advancements, LLMs still encounter challenges with length control, frequently failing to adhere to specific length constraints due to their token-level operations and insufficient training on data with strict length limitations. We identify this issue as stemming from a lack of positional awareness and propose novel approaches-PositionID Prompting and Posi-tionID Fine-Tuning-to address it. These methods enhance the model's ability to continuously monitor and manage text length during generation. Additionally, we introduce PositionID CP Prompting to enable LLMs to perform copy and paste operations accurately. Furthermore, we develop two benchmarks for evaluating length control and copy-paste abilities. Our experiments demonstrate that our methods significantly improve the model's adherence to length constraints and copy-paste accuracy without compromising response quality. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. From Sub-Ability Diagnosis to Human-Aligned Generation: Bridging the Gap for Text Length Control via MarkerGenPeiwen Yuan, Chuyi Tan, Shaoxiong Feng, Yiwei Li et al.ACL 2025 · 2 citations
  2. Length Controlled Generation for Black-box LLMsYuxuan Gu, Wenjie Wang, Xiaocheng Feng, Weihong Zhong et al.ACL 2025

Builds on3

  1. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu et al.ICLR 2024 · 1,469 citations
  2. COLD Decoding: Energy-based Constrained Text Generation with Langevin DynamicsLianhui Qin, Sean Welleck, Daniel Khashabi, Yejin ChoiNeurIPS 2022 · 217 citations
  3. Transformers Can Do Arithmetic with the Right EmbeddingsSean McLeish, Arpit Bansal, Alex Stein, Neel Jain et al.NeurIPS 2024 · 94 citations

Related papers

  1. PIC: Unlocking Long-Form Text Generation Capabilities of Large Language Models via Position ID CompressionHaoran Que, Wenge RongACL 2025 · 2 citations
  2. LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language ModelsWei Zhang, Lintong Du, yuanhe zhang, Zhenhong Zhou et al.ICML 2026
  3. Ruler: A Model-Agnostic Method to Control Generated Length for Large Language ModelsJiaming Li, Lei Zhang, Yunshui Li, Ziqiang Liu et al.EMNLP Findings 2024 · 1 citation
  4. DcLM: Output Length Control of Large Language Models via Dynamic Length MarkersZhe Chen, Jiaao Yu, Honglin LiACL Findings 2026
  5. Prompt-Based Length Controlled Generation with Multiple Control TypesRenlong Jie, Xiaojun Meng, Lifeng Shang, Xin Jiang et al.ACL Findings 2024 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines