Text Has Curvature
Karish Grover, Hanqing Zeng, Yinglong Xia, Christos Faloutsos, Geoff Gordon
摘要
Does natural language text have an intrinsic curvature? Language is increasingly modeled in curved geometries—hyperbolic spaces for hierarchy, mixed-curvature manifolds for compositional structure—yet a basic scientific question remains unresolved: what does curvature mean for text itself, in a way that is native to language rather than an artifact of the embedding space we choose? We argue that text does indeed have curvature, and show how to detect it, define it, and use it. To this end, we propose Texture, a text-native, word-level discrete curvature signal, and make three contributions. (a) Existence: We provide empirical and theoretical certificates that semantic inference in natural corpora is non-flat. (b) Definition: We define Texture as a signed two-axis curvature of the word-in-context belief field—the differential of reconciliation between prefix and suffix—measuring, via a debiased Schrödinger transport divergence, whether adding context from one side contracts the semantic effect of context from the other side (focus, positive) or expands it into competing continuations (fan-out, negative). (c) Utility: Texture is actionable: it serves as a general-purpose measurement and control primitive enabling geometry without geometric training; we instantiate it on two representative tasks, improving long-context inference through curvature-guided compression and retrieval-augmented generation through curvature-guided routing. Together, our results establish a text native curvature paradigm, making Texture practically useful.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper6
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni 等NeurIPS 2020 · 被引用 19,162 次
- Hyperbolic Neural Networks++Ryohei Shimizu, Yusuke Mukuta, Tatsuya HaradaICLR 2021 · 被引用 791 次
- Active Retrieval Augmented GenerationZhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun 等EMNLP 2023 · 被引用 315 次
- LongBench: A Bilingual, Multitask Benchmark for Long Context UnderstandingYushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu 等ACL 2024 · 被引用 94 次
- Compressing Context to Enhance Inference Efficiency of Large Language ModelsYucheng Li, Bo Dong, Frank Guerin, Chenghua LinEMNLP 2023 · 被引用 54 次
相关 Paper
- Representational Curvature Modulates Behavioral Uncertainty in Large Language ModelsJack King, Evelina Fedorenko, Eghbal HosseiniICML 2026
- DMAP: A Distribution Map for TextTom Kempton, Julia Rozanova, Parameswaran Kamalaruban, Maeve Madigan 等ICLR 2026 · 被引用 1 次
- HELM: Hyperbolic Large Language Models via Mixture-of-Curvature ExpertsNeil He, Rishabh Anand, Hiren Madhu, Ali Maatouk 等NeurIPS 2025 · 被引用 27 次
- Correlation Dimension of Autoregressive Large Language ModelsXin Du, Kumiko Tanaka-IshiiNeurIPS 2025 · 被引用 2 次
- HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented GenerationHiren Madhu, Ngoc Bui, Ali Maatouk, Leandros Tassiulas 等ICML 2026 · 被引用 1 次
