ACL Findings2026
Progress Ratio Embeddings: An Impatience Signal for Robust Length Control in Neural Text Generation
Ivanhoé Botcazou, Tassadit Amghar, Sylvain Lamprier, Frédéric Saubion
Abstract
Modern neural language models achieve high accuracy in text generation, yet precise control over generation length remains underdeveloped. In this paper, we first investigate a recent length control method based on Reverse Positional Embeddings (RPE) and show its limits when control is requested beyond the training distribution. In particular, using a discrete countdown signal tied to the absolute remaining token count leads to instability. To provide robust length control, we introduce Progress Ratio Embeddings (PRE), as continuous embeddings tied to a trigonometric impatience signal. PRE integrates seamlessly into standard Transformer architectures, providing stable length fidelity without degrading text accuracy under standard evaluation metrics. We further show that PRE generalizes well to unseen target lengths. Experiments on two widely used news-summarization benchmarks and a popular question generation dataset validate these findings. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- LenAtten: An Effective Length Controlling Unit For Text Summarization cites BERTScore: Evaluating Text Generation with BERT
- Length Control in Abstractive Summarization by Pretraining Information Selection cites BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
- Length Control in Abstractive Summarization by Pretraining Information Selection cites LenAtten: An Effective Length Controlling Unit For Text Summarization
Builds on7
- BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- A Survey on In-context LearningQingxiu Dong, Lei Li, Damai Dai, Ce Zheng et al.EMNLP 2024 · 479 citations
- Length Control in Abstractive Summarization by Pretraining Information SelectionYizhu Liu, Qi Jia, Kenny Q. ZhuACL 2022 · 39 citations
- LenAtten: An Effective Length Controlling Unit For Text SummarizationZhongyi Yu, Zhenghao Wu, Hao Zheng, Zhe Xuanyuan et al.ACL Findings 2021 · 13 citations
Related papers
- Extending the Context of Pretrained LLMs by Dropping Their Positional EmbeddingYoav Gelberg, Koshi Eguchi, Takuya Akiba, Edoardo CetinICLR 2026 · 13 citations
- DcLM: Output Length Control of Large Language Models via Dynamic Length MarkersZhe Chen, Jiaao Yu, Honglin LiACL Findings 2026
- The Role of Sparsity for Length Generalization in LLMsNoah Golowich, Samy Jelassi, David Brandfonbrener, Sham M. Kakade et al.ICML 2025
- Prompt-Based Length Controlled Generation with Multiple Control TypesRenlong Jie, Xiaojun Meng, Lifeng Shang, Xin Jiang et al.ACL Findings 2024 · 4 citations
- Functional Interpolation for Relative Positions improves Long Context TransformersShanda Li, Chong You, Guru Guruganesh, Joshua Ainslie et al.ICLR 2024 · 66 citations
