Lune

ACL Findings2025

READoc: A Unified Benchmark for Realistic Document Structured Extraction

Zichao Li, Aizier Abulaiti, Yaojie Lu, Xuanang Chen, Jia Zheng, Hongyu Lin, Xianpei Han, Shanshan Jiang, Bin Dong, Le Sun

Abstract

Document Structured Extraction (DSE) aims to extract structured content from raw documents.Despite the emergence of numerous DSE systems, their unified evaluation remains inadequate, significantly hindering the field's advancement.This problem is largely attributed to existing benchmark paradigms, which exhibit fragmented and localized characteristics.To offer a thorough evaluation of DSE systems, we introduce a novel benchmark named READOC, which defines DSE as a realistic task of converting unstructured PDFs into semantically rich Markdown.The READOC dataset is derived from 3,576 diverse and real-world documents from arXiv, GitHub, and Zenodo.In addition, we develop a DSE Evaluation S 3 uite comprising Standardization, Segmentation and Scoring modules, to conduct a unified evaluation of state-of-theart DSE approaches.By evaluating a range of pipeline tools, expert visual models, and general Vision-Language Models, we identify the gap between current work and the unified, realistic DSE objective for the first time.We aspire that READOC will catalyze future research in DSE, fostering more comprehensive and practical solutions.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking cites LayoutLM: Pre-training of Text and Layout for Document Image Understanding
  • LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking cites DocFormer: End-to-End Transformer for Document Understanding
  • DocFormer: End-to-End Transformer for Document Understanding cites LayoutLM: Pre-training of Text and Layout for Document Image Understanding
  • Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding cites LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
  • Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding cites DocFormer: End-to-End Transformer for Document Understanding
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented GenerationJunyuan Zhang, Qintong Zhang, Bin Wang, Linke Ouyang et al.ICCV 2025 · 10 citations
  2. XtraGPT: Context-Aware and Controllable Academic Paper Revision via Human-AI CollaborationNuo Chen, Andre Huikai Lin, Jiaying Wu, Junyi Hou et al.ACL 2026 · 3 citations
  3. TexOCR: Advancing Document OCR Models for Compilable Page-to-LaTeX ReconstructionChengye Wang, Lin Fu, Zexi Kuang, Yilun ZhaoACL 2026
  4. MessToClean: Evidence-Grounded Structure-Preserving Reconstruction for Real-World Degraded Exam Paper ImagesJiayi Tuo, Cheng Tang, Zihan Wang, Chenyue Zhou et al.ACL 2026

Builds on16

  1. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  2. LayoutLMv3: Pre-training for Document AI with Unified Text and Image MaskingYupan Huang, Tengchao Lv, Lei Cui, Yutong Lu et al.ACM MM 2022 · 606 citations
  3. LayoutLM: Pre-training of Text and Layout for Document Image UnderstandingYiheng Xu, Minghao Li, Lei Cui, Shaohan Huang et al.KDD 2020 · 575 citations
  4. Pix2Struct: Screenshot Parsing as Pretraining for Visual Language UnderstandingKenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu et al.ICML 2023 · 426 citations
  5. DocFormer: End-to-End Transformer for Document UnderstandingSrikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie et al.ICCV 2021 · 392 citations

Related papers

  1. OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive AnnotationsLinke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu et al.CVPR 2025
  2. Document Understanding Dataset and Evaluation (DUDE)Jordy Van Landeghem, Rafal Powalski, Rubèn Tito, Dawid Jurkiewicz et al.ICCV 2023 · 130 citations
  3. MMDocIR: Benchmarking Multimodal Retrieval for Long DocumentsKuicai Dong, Yujing Chang, Derrick-Goh-Xin Deik, Dexun Li et al.EMNLP 2025 · 1 citation
  4. mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document UnderstandingAnwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan et al.EMNLP Findings 2024 · 60 citations
  5. Modeling Layout Reading Order as Ordering Relations for Visually-rich Document UnderstandingChong Zhang, Yi Tu, Yixi Zhao, Chenshu Yuan et al.EMNLP 2024 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines