Text Has Curvature
Karish Grover, Hanqing Zeng, Yinglong Xia, Christos Faloutsos, Geoff Gordon
Abstract
Does natural language text have an intrinsic curvature? Language is increasingly modeled in curved geometries—hyperbolic spaces for hierarchy, mixed-curvature manifolds for compositional structure—yet a basic scientific question remains unresolved: what does curvature mean for text itself, in a way that is native to language rather than an artifact of the embedding space we choose? We argue that text does indeed have curvature, and show how to detect it, define it, and use it. To this end, we propose Texture, a text-native, word-level discrete curvature signal, and make three contributions. (a) Existence: We provide empirical and theoretical certificates that semantic inference in natural corpora is non-flat. (b) Definition: We define Texture as a signed two-axis curvature of the word-in-context belief field—the differential of reconciliation between prefix and suffix—measuring, via a debiased Schrödinger transport divergence, whether adding context from one side contracts the semantic effect of context from the other side (focus, positive) or expands it into competing continuations (fan-out, negative). (c) Utility: Texture is actionable: it serves as a general-purpose measurement and control primitive enabling geometry without geometric training; we instantiate it on two representative tasks, improving long-context inference through curvature-guided compression and retrieval-augmented generation through curvature-guided routing. Together, our results establish a text native curvature paradigm, making Texture practically useful.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on6
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Hyperbolic Neural Networks++Ryohei Shimizu, Yusuke Mukuta, Tatsuya HaradaICLR 2021 · 791 citations
- Active Retrieval Augmented GenerationZhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun et al.EMNLP 2023 · 315 citations
- LongBench: A Bilingual, Multitask Benchmark for Long Context UnderstandingYushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu et al.ACL 2024 · 94 citations
- Compressing Context to Enhance Inference Efficiency of Large Language ModelsYucheng Li, Bo Dong, Frank Guerin, Chenghua LinEMNLP 2023 · 54 citations
Related papers
- Representational Curvature Modulates Behavioral Uncertainty in Large Language ModelsJack King, Evelina Fedorenko, Eghbal HosseiniICML 2026
- DMAP: A Distribution Map for TextTom Kempton, Julia Rozanova, Parameswaran Kamalaruban, Maeve Madigan et al.ICLR 2026 · 1 citation
- HELM: Hyperbolic Large Language Models via Mixture-of-Curvature ExpertsNeil He, Rishabh Anand, Hiren Madhu, Ali Maatouk et al.NeurIPS 2025 · 27 citations
- Correlation Dimension of Autoregressive Large Language ModelsXin Du, Kumiko Tanaka-IshiiNeurIPS 2025 · 2 citations
- HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented GenerationHiren Madhu, Ngoc Bui, Ali Maatouk, Leandros Tassiulas et al.ICML 2026 · 1 citation
