An Inclusive Notion of Text
Ilia Kuznetsov, Iryna Gurevych
Abstract
Natural language processing (NLP) researchers develop models of grammar, meaning and communication based on written text. Due to task and data differences, what is considered text can vary substantially across studies. A conceptual framework for systematically capturing these differences is lacking. We argue that clarity on the notion of text is crucial for reproducible and generalizable NLP. Towards that goal, we propose common terminology to discuss the production and transformation of textual data, and introduce a two-tier taxonomy of linguistic and non-linguistic elements that are available in textual sources and can be used in NLP modeling. We apply this taxonomy to survey existing work that extends the notion of text beyond the conservative language-centered view. We outline key desiderata and challenges of the emerging inclusive approach to text in NLP, and suggest community-level reporting as a crucial next step to consolidate the discussion.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext ecb2a0b0-61d1-457d-8e0e-9fb59ee87d73Builds on6
- S2ORC: The Semantic Scholar Open Research CorpusKyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney et al.ACL 2020 · 424 citations
- HTLM: Hyper-Text Pre-Training and Prompting of Language ModelsArmen Aghajanyan, Dmytro Okhonko, Mike Lewis, Mandar Joshi et al.ICLR 2022 · 82 citations
- MarkupLM: Pre-training of Text and Markup Language for Visually Rich Document UnderstandingJunlong Li, Yiheng Xu, Lei Cui, Furu WeiACL 2022 · 75 citations
- SPECTER: Document-level Representation Learning using Citation-informed TransformersArman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey et al.ACL 2020 · 20 citations
- Argument Pair Extraction with Mutual Guidance and Inter-sentence Relation GraphJianzhu Bao, Bin Liang, Jingyi Sun, Yice Zhang et al.EMNLP 2021 · 14 citations
Related papers
- Disentangling syntax and semantics in the brain with deep networksCharlotte Caucheteux, Alexandre Gramfort, Jean-Remi KingICML 2021 · 96 citations
- The Nature of NLP: Analyzing Contributions in NLP PapersAniket Pramanick, Yufang Hou, Saif M. Mohammad, Iryna GurevychACL 2025 · 9 citations
- NLP Reproducibility For All: Understanding Experiences of BeginnersShane Storks, Keunwoo Peter Yu, Ziqiao Ma, Joyce ChaiACL 2023
- Systematic Task Exploration with LLMs: A Study in Citation Text GenerationFurkan Sahinuç, Ilia Kuznetsov, Yufang Hou, Iryna GurevychACL 2024
- Toward Compositional Behavior in Neural Models: A Survey of Current ViewsKate McCurdy, Paul Soulos, Paul Smolensky, Roland Fernandez et al.EMNLP 2024 · 12 citations
