OvisOCR: End-to-End Document Parsing via Aligning Specialized Perception with General Reasoning
Jun-Peng Jiang, Shiyin Lu, An-Yang Ji, Yinglun Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, Han-Jia Ye
Abstract
This paper presents OvisOCR, a lightweight and strictly end-to-end Multimodal Large Language Model (MLLM) tailored for document parsing. Unlike current methods that rely on complex "Crop-OCR-Merge'" cascades to handle high-resolution inputs, OvisOCR directly maps full-page visual signals to structured Markdown without localized slicing or layout detection dependencies. Through extensive evaluations on the OmniDocBench, OvisOCR achieves top-tier performance among all compared methods in all aspects, demonstrating that a compact E2E model can effectively ``digest'' the capabilities of intricate pipelines and surpass specialized and general methods. Technically, OvisOCR unifies fine-grained recognition and semantic correction in a single model by leveraging supervision bootstrapped from strong OCR engines and refined via general-model-based data cleaning. To balance the performance across diverse document constituents, we design category-specific reward mechanisms for distinct element types, such as dense text, complex tables, and formulas, and ensure the model enhances its formatting strengths for each modality concurrently. This approach effectively resolves the optimization conflict, guaranteeing that improvements in structural layout parsing do not come at the expense of omitting fine-grained textual details. Empirical results confirm that OvisOCR eliminates the error propagation inherent in split-and-merge architectures, offering a streamlined path for future document intelligence. Our model is available at https://huggingface.co/ATH-MaaS/OvisOCR.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on7
- Real-Time Scene Text Detection with Differentiable BinarizationMinghui Liao, Zhaoyi Wan, Cong Yao, Kai Chen et al.AAAI 2020 · 818 citations
- Nougat: Neural Optical Understanding for Academic DocumentsLukas Blecher, Guillem Cucurull, Thomas Scialom, Robert StojnicICLR 2024 · 243 citations
- GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL OptimizationShih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao et al.ICML 2026 · 128 citations
- VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning贾 子怡, Zijian Cheng, Xinyue Zhang, Kun-Yang Yu et al.ICML 2026 · 3 citations
- OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive AnnotationsLinke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu et al.CVPR 2025
Related papers
- Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware TrainingGengluo Li, Pengyuan Lyu, Chengquan Zhang, Huawen Shen et al.CVPR 2026 · 9 citations
- DocVLM: Make Your VLM an Efficient ReaderMor Shpigel Nacson, Aviad Aberdam, Roy Ganz, Elad Ben-Avraham et al.CVPR 2025
- UniDocVLM: Enhancing Visual Reasoning and Document Understanding for VLM via Reinforcement LearningZongsheng Cao, Anran Liu, Jun Xie, Lang Chen et al.KDD 2026
- DocLLM: A Layout-Aware Generative Language Model for Multimodal Document UnderstandingDongsheng Wang, Natraj Raman, Mathieu Sibue, Zhiqiang Ma et al.ACL 2024 · 37 citations
- CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in LiteracyZhibo Yang, Jun Tang, Zhaohai Li, Pengfei Wang et al.ICCV 2025 · 15 citations
