Lune

ACL Findings2023

GVdoc - Graph-based Visual DOcument Classification

Fnu Mohbat, Mohammed J. Zaki, Catherine Finegan-Dollak, Ashish Verma

Abstract

The robustness of a model for real-world deployment is decided by how well it performs on unseen data and distinguishes between in-domain and out-of-domain samples. Visual document classifiers have shown impressive performance on in-distribution test sets. However, they tend to have a hard time correctly classifying and differentiating out-ofdistribution examples. Image-based classifiers lack the text component, whereas multimodality transformer-based models face the token serialization problem in visual documents due to their diverse layouts. They also require a lot of computing power during inference, making them impractical for many real-world applications. We propose, GVdoc, a graph-based document classification model that addresses both of these challenges. Our approach generates a document graph based on its layout, and then trains a graph neural network to learn node and graph embeddings. Through experiments, we show that our model, even with fewer parameters, outperforms state-of-the-art models on out-of-distribution data while retaining comparable performance on the in-distribution test set.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking cites LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding
  • LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking cites BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents
  • LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking cites DiT: Self-supervised Pre-training for Document Image Transformer
  • DiT: Self-supervised Pre-training for Document Image Transformer cites LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
  • Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding cites LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
  • Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding cites BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Modeling Layout Reading Order as Ordering Relations for Visually-rich Document UnderstandingChong Zhang, Yi Tu, Yixi Zhao, Chenshu Yuan et al.EMNLP 2024 · 4 citations

Builds on12

  1. Energy-based Out-of-distribution DetectionWeitang Liu, Xiaoyun Wang, John D. Owens, Yixuan LiNeurIPS 2020 · 2,213 citations
  2. LayoutLMv3: Pre-training for Document AI with Unified Text and Image MaskingYupan Huang, Tengchao Lv, Lei Cui, Yutong Lu et al.ACM MM 2022 · 606 citations
  3. LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document UnderstandingJiapeng Wang, Lianwen Jin, Kai DingACL 2022 · 188 citations
  4. BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from DocumentsTeakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang et al.AAAI 2022 · 186 citations
  5. DiT: Self-supervised Pre-training for Document Image TransformerJunlong Li, Yiheng Xu, Tengchao Lv, Lei Cui et al.ACM MM 2022 · 184 citations

Related papers

  1. Contrastive Document Representation Learning with Graph Attention NetworksPeng Xu, Xinchi Chen, Xiaofei Ma, Zhiheng Huang et al.EMNLP Findings 2021 · 10 citations
  2. VecFormer: Towards Efficient and Generalizable Graph Transformer with Graph Token AttentionJingbo Zhou, Jun Xia, Siyuan Li, Yunfan Liu et al.WWW 2026
  3. Beyond Text: Incorporating Metadata and Label Structure for Multi-Label Document Classification using Heterogeneous GraphsChenchen Ye, Linhai Zhang, Yulan He, Deyu Zhou et al.EMNLP 2021 · 9 citations
  4. Sparse Structure Learning via Graph Neural Networks for Inductive Document ClassificationYinhua Piao, Sangseon Lee, Dohoon Lee, Sun KimAAAI 2022 · 46 citations
  5. DocFormer: End-to-End Transformer for Document UnderstandingSrikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie et al.ICCV 2021 · 392 citations
Dusk over the sea between two cliffs drawn in fine vertical lines