EMNLP Findings2023
Connecting the Dots: What Graph-Based Text Representations Work Best for Text Classification using Graph Neural Networks?
Abstract
Given the success of Graph Neural Networks (GNNs) for structure-aware machine learning, many studies have explored their use for text classification, but mostly in specific domains with limited data characteristics.Moreover, some strategies prior to GNNs relied on graph mining and classical machine learning, making it difficult to assess their effectiveness in modern settings.This work extensively investigates graph representation methods for text classification, identifying practical implications and open challenges.We compare different graph construction schemes using a variety of GNN architectures and setups across five datasets, encompassing short and long documents as well as unbalanced scenarios in diverse domains.Two Transformer-based large language models are also included to complement the study.The results show that i) although the effectiveness of graphs depends on the textual input features and domain, simple graph constructions perform better the longer the documents are, ii) graph representations are especially beneficial for longer documents, outperforming Transformer-based models, iii) graph methods are particularly efficient at solving the task. WindowWindowext Sequence Sequence simp Emb. Layers Units Acc F1-ma Acc F1-ma Acc
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Be More with Less: Hypergraph Attention Networks for Inductive Text Classification cites Tensor Graph Convolutional Networks for Text Classification
- Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLP cites Be More with Less: Hypergraph Attention Networks for Inductive Text Classification
- Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLP cites Tensor Graph Convolutional Networks for Text Classification
- Graph with Sequence: Broad-Range Semantic Modeling for Fake News Detection cites Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLP
- Graph with Sequence: Broad-Range Semantic Modeling for Fake News Detection cites Message Passing Attention Networks for Document Understanding
Cited by top-tier papers2
- NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long DocumentsTamara Czinczoll, Christoph Hönes, Maximilian Schall, Gerard de MeloACL 2024 · 4 citations
- Graph with Sequence: Broad-Range Semantic Modeling for Fake News DetectionJunwei Yin, Min Gao, Kai Shu, Wentao Li et al.WWW 2025 · 4 citations
Builds on4
- Tensor Graph Convolutional Networks for Text ClassificationXien Liu, Xinxin You, Xiao Zhang, Ji Wu et al.AAAI 2020 · 284 citations
- Be More with Less: Hypergraph Attention Networks for Inductive Text ClassificationKaize Ding, Jianling Wang, Jundong Li, Dingcheng Li et al.EMNLP 2020 · 210 citations
- Message Passing Attention Networks for Document UnderstandingGiannis Nikolentzos, Antoine J.-P. Tixier, Michalis VazirgiannisAAAI 2020 · 80 citations
- Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLPLukas Galke, Ansgar ScherpACL 2022
Related papers
- Contrastive Document Representation Learning with Graph Attention NetworksPeng Xu, Xinchi Chen, Xiaofei Ma, Zhiheng Huang et al.EMNLP Findings 2021 · 10 citations
- Deep Attention Diffusion Graph Neural Networks for Text ClassificationYonghao Liu, Renchu Guan, Fausto Giunchiglia, Yanchun Liang et al.EMNLP 2021 · 67 citations
- Contrastive Learning with Simplicial Convolutional Networks for Short-Text ClassificationHuang Liang, Benedict Lee, Daniel Hui Loong Ng, Kelin XiaICML 2025
- Sparse Structure Learning via Graph Neural Networks for Inductive Document ClassificationYinhua Piao, Sangseon Lee, Dohoon Lee, Sun KimAAAI 2022 · 46 citations
- Do Transformers Really Perform Badly for Graph Representation?Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng et al.NeurIPS 2021 · 1,632 citations
