ACL Findings2023
GVdoc - Graph-based Visual DOcument Classification
Fnu Mohbat, Mohammed J. Zaki, Catherine Finegan-Dollak, Ashish Verma
Abstract
The robustness of a model for real-world deployment is decided by how well it performs on unseen data and distinguishes between in-domain and out-of-domain samples. Visual document classifiers have shown impressive performance on in-distribution test sets. However, they tend to have a hard time correctly classifying and differentiating out-ofdistribution examples. Image-based classifiers lack the text component, whereas multimodality transformer-based models face the token serialization problem in visual documents due to their diverse layouts. They also require a lot of computing power during inference, making them impractical for many real-world applications. We propose, GVdoc, a graph-based document classification model that addresses both of these challenges. Our approach generates a document graph based on its layout, and then trains a graph neural network to learn node and graph embeddings. Through experiments, we show that our model, even with fewer parameters, outperforms state-of-the-art models on out-of-distribution data while retaining comparable performance on the in-distribution test set.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking cites LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding
- LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking cites BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents
- LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking cites DiT: Self-supervised Pre-training for Document Image Transformer
- DiT: Self-supervised Pre-training for Document Image Transformer cites LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
- Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding cites LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
- Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding cites BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents
Cited by top-tier papers1
Builds on12
- Energy-based Out-of-distribution DetectionWeitang Liu, Xiaoyun Wang, John D. Owens, Yixuan LiNeurIPS 2020 · 2,213 citations
- LayoutLMv3: Pre-training for Document AI with Unified Text and Image MaskingYupan Huang, Tengchao Lv, Lei Cui, Yutong Lu et al.ACM MM 2022 · 606 citations
- LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document UnderstandingJiapeng Wang, Lianwen Jin, Kai DingACL 2022 · 188 citations
- BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from DocumentsTeakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang et al.AAAI 2022 · 186 citations
- DiT: Self-supervised Pre-training for Document Image TransformerJunlong Li, Yiheng Xu, Tengchao Lv, Lei Cui et al.ACM MM 2022 · 184 citations
Related papers
- Contrastive Document Representation Learning with Graph Attention NetworksPeng Xu, Xinchi Chen, Xiaofei Ma, Zhiheng Huang et al.EMNLP Findings 2021 · 10 citations
- VecFormer: Towards Efficient and Generalizable Graph Transformer with Graph Token AttentionJingbo Zhou, Jun Xia, Siyuan Li, Yunfan Liu et al.WWW 2026
- Beyond Text: Incorporating Metadata and Label Structure for Multi-Label Document Classification using Heterogeneous GraphsChenchen Ye, Linhai Zhang, Yulan He, Deyu Zhou et al.EMNLP 2021 · 9 citations
- Sparse Structure Learning via Graph Neural Networks for Inductive Document ClassificationYinhua Piao, Sangseon Lee, Dohoon Lee, Sun KimAAAI 2022 · 46 citations
- DocFormer: End-to-End Transformer for Document UnderstandingSrikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie et al.ICCV 2021 · 392 citations
