Lune

EMNLP Findings2025

Intelligent Document Parsing: Towards End-to-end Document Parsing via Decoupled Content Parsing and Layout Grounding

Hangdi Xing, Feiyu Gao, Qi Zheng, Zhaoqing Zhu, Zirui Shao, Ming Yan

Abstract

In the daily work, vast amounts of documents are stored in pixel-based formats such as images and scanned PDFs, posing challenges for efficient database management and data processing. Existing methods often fragment the parsing process into the pipeline of separated subtasks on the layout element level, resulting in incomplete semantics and error propagation. Even though models based on multimodal large language models (MLLMs) mitigate the issues to some extent, they also suffer from absent or sub-optimal grounding ability for visual information. To address these challenges, we introduce the Intelligent Document Parsing (IDP) framework, an end-to-end document parsing framework leveraging the visionlanguage priors of MLLMs, equipped with an elaborately designed document representation and decoding mechanism to decouple the content parsing and layout grounding to fully activate the potential of MLLMs for document parsing. Experimental results demonstrate that the IDP method surpasses existing methods, significantly advancing MLLM-based document parsing.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Nougat: Neural Optical Understanding for Academic Documents cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
  • VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks cites ViperGPT: Visual Inference via Python Execution for Reasoning
  • VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Embodied Task Planning via Graph-Informed Action Generation with Large Language ModelsXiang Li, Ning Yan, Masood MortazaviICML 2026 · 2 citations
  2. M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language ModelsJoongmin Shin, Jeongbae Park, Jaehyung Seo, Heuiseok LimCVPR 2026
  3. MessToClean: Evidence-Grounded Structure-Preserving Reconstruction for Real-World Degraded Exam Paper ImagesJiayi Tuo, Cheng Tang, Zihan Wang, Chenyue Zhou et al.ACL 2026

Builds on18

  1. Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsZe Liu, Yutong Lin, Yue Cao, Han Hu et al.ICCV 2021 · 31,683 citations
  2. DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingFeng Li, Hao Zhang, Shilong Liu, Jian Guo et al.CVPR 2022 · 879 citations
  3. ViperGPT: Visual Inference via Python Execution for ReasoningDídac Surís, Sachit Menon, Carl VondrickICCV 2023 · 732 citations
  4. Nougat: Neural Optical Understanding for Academic DocumentsLukas Blecher, Guillem Cucurull, Thomas Scialom, Robert StojnicICLR 2024 · 243 citations
  5. VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language TasksJiannan Wu, Muyan Zhong, Sen Xing, Zeqiang Lai et al.NeurIPS 2024 · 179 citations

Related papers

  1. Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware TrainingGengluo Li, Pengyuan Lyu, Chengquan Zhang, Huawen Shen et al.CVPR 2026 · 9 citations
  2. A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging TrendsYihao Ding, Siwen Luo, Yue Dai, Yanbei Jiang et al.ACL Findings 2026 · 4 citations
  3. LMDX: Language Model-based Document Information Extraction and LocalizationVincent Perot, Kai Kang, Florian Luisier, Guolong Su et al.ACL Findings 2024 · 38 citations
  4. DocHieNet: A Large and Diverse Dataset for Document Hierarchy ParsingHangdi Xing, Changxu Cheng, Feiyu Gao, Zirui Shao et al.EMNLP 2024 · 3 citations
  5. VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality DocumentsShi Yu, Chaoyue Tang, Bokai Xu, Junbo Cui et al.ICLR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines