The Harmonic Structure of Information Contours
Eleftheria Tsipidi, Samuel Kiegeland, Franz Nowak, Tianyang Xu, Ethan Wilcox, Alex Warstadt, Ryan Cotterell, Mario Giulianelli
Abstract
The uniform information density (UID) hypothesis proposes that speakers aim to distribute information evenly throughout a text, balancing production effort and listener comprehension difficulty. However, language typically does not maintain a strictly uniform information rate; instead, it fluctuates around a global average. These fluctuations are often explained by factors such as syntactic constraints, stylistic choices, or audience design. In this work, we explore an alternative perspective: that these fluctuations may be influenced by an implicit linguistic pressure towards periodicity, where the information rate oscillates at regular intervals, potentially across multiple frequencies simultaneously. We apply harmonic regression and introduce a novel extension called time scaling to detect and test for such periodicity in information contours. Analyzing texts in English, Spanish, German, Dutch, Basque, and Brazilian Portuguese, we find consistent evidence of periodic patterns in information rate. Many dominant frequencies align with discourse structure, suggesting these oscillations reflect meaningful linguistic organization. Beyond highlighting the connection between information rate and discourse structure, our approach offers a general framework for uncovering structural pressures at various levels of linguistic granularity.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 97db1d89-97e9-4a1f-b25d-27823a6bab83Cited by top-tier papers3
- Identifying the Periodicity of Information in Natural LanguageYulin Ou, Yu Wang, Yang Xu, Hendrik BuschmeierACL 2026 · 2 citations
- Surprisal Minimisation over Goal-directed Alternatives Predicts Production Choice in DialogueThomas P. Utting, Mario Giulianelli, Arabella SinclairACL 2026
- On the Proper Treatment of Units in Surprisal TheorySamuel Kiegeland, Vésteinn Snæbjarnarson, Tim Vieira, Ryan CotterellACL 2026
Builds on10
- YaRN: Efficient Context Window Extension of Large Language ModelsBowen Peng, Jeffrey Quesnelle, Honglu Fan, Enrico ShippoleICLR 2024 · 508 citations
- Detecting Subtle Differences between Human and Model Languages Using Spectrum of Relative LikelihoodYang Xu, Yu Wang, Hao An, Zhichen Liu et al.EMNLP 2024 · 20 citations
- FACE: Evaluating Natural Language Generation with Fourier Analysis of Cross-EntropyZuhao Yang, Yingfang Yuan, Yang Xu, Shuo Zhan et al.NeurIPS 2023 · 11 citations
- Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence SegmentationMarkus Frohmann, Igor Sterner, Ivan Vulic, Benjamin Minixhofer et al.EMNLP 2024 · 10 citations
- Where's the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence SegmentationBenjamin Minixhofer, Jonas Pfeiffer, Ivan VulicACL 2023 · 8 citations
Related papers
- Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form DiscourseEleftheria Tsipidi, Franz Nowak, Ryan Cotterell, Ethan Wilcox et al.EMNLP 2024 · 2 citations
- Revisiting the Uniform Information Density HypothesisClara Meister, Tiago Pimentel, Patrick Haller, Lena A. Jäger et al.EMNLP 2021 · 4 citations
- A Cognitive Regularizer for Language ModelingJason Wei, Clara Meister, Ryan CotterellACL 2021
- Uniform Information Density and Syntactic Reduction: Revisiting that-Mentioning in English Complement ClausesHailin Hao, Elsi KaiserEMNLP 2025
- Expect the Unexpected? Testing the Surprisal of Salient EntitiesJessica Lin, Amir ZeldesACL 2026 · 2 citations
