OvisOCR: End-to-End Document Parsing via Aligning Specialized Perception with General Reasoning
Jun-Peng Jiang, Shiyin Lu, An-Yang Ji, Yinglun Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, Han-Jia Ye
摘要
This paper presents OvisOCR, a lightweight and strictly end-to-end Multimodal Large Language Model (MLLM) tailored for document parsing. Unlike current methods that rely on complex "Crop-OCR-Merge'" cascades to handle high-resolution inputs, OvisOCR directly maps full-page visual signals to structured Markdown without localized slicing or layout detection dependencies. Through extensive evaluations on the OmniDocBench, OvisOCR achieves top-tier performance among all compared methods in all aspects, demonstrating that a compact E2E model can effectively ``digest'' the capabilities of intricate pipelines and surpass specialized and general methods. Technically, OvisOCR unifies fine-grained recognition and semantic correction in a single model by leveraging supervision bootstrapped from strong OCR engines and refined via general-model-based data cleaning. To balance the performance across diverse document constituents, we design category-specific reward mechanisms for distinct element types, such as dense text, complex tables, and formulas, and ensure the model enhances its formatting strengths for each modality concurrently. This approach effectively resolves the optimization conflict, guaranteeing that improvements in structural layout parsing do not come at the expense of omitting fine-grained textual details. Empirical results confirm that OvisOCR eliminates the error propagation inherent in split-and-merge architectures, offering a streamlined path for future document intelligence. Our model is available at https://huggingface.co/ATH-MaaS/OvisOCR.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper7
- Real-Time Scene Text Detection with Differentiable BinarizationMinghui Liao, Zhaoyi Wan, Cong Yao, Kai Chen 等AAAI 2020 · 被引用 818 次
- Nougat: Neural Optical Understanding for Academic DocumentsLukas Blecher, Guillem Cucurull, Thomas Scialom, Robert StojnicICLR 2024 · 被引用 243 次
- GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL OptimizationShih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao 等ICML 2026 · 被引用 128 次
- VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning贾 子怡, Zijian Cheng, Xinyue Zhang, Kun-Yang Yu 等ICML 2026 · 被引用 3 次
- OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive AnnotationsLinke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu 等CVPR 2025
相关 Paper
- Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware TrainingGengluo Li, Pengyuan Lyu, Chengquan Zhang, Huawen Shen 等CVPR 2026 · 被引用 9 次
- DocVLM: Make Your VLM an Efficient ReaderMor Shpigel Nacson, Aviad Aberdam, Roy Ganz, Elad Ben-Avraham 等CVPR 2025
- UniDocVLM: Enhancing Visual Reasoning and Document Understanding for VLM via Reinforcement LearningZongsheng Cao, Anran Liu, Jun Xie, Lang Chen 等KDD 2026
- DocLLM: A Layout-Aware Generative Language Model for Multimodal Document UnderstandingDongsheng Wang, Natraj Raman, Mathieu Sibue, Zhiqiang Ma 等ACL 2024 · 被引用 37 次
- CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in LiteracyZhibo Yang, Jun Tang, Zhaohai Li, Pengfei Wang 等ICCV 2025 · 被引用 15 次
