Dissecting Post-Training: Uncovering the Complementary Roles of SFT and RL for Document Parsing
Jun-Peng Jiang, An-Yang Ji, Shiyin Lu, Guodong Zheng, Weihong Zhang, Qing-Guo Chen, Weihua Luo, Kaifu Zhang, Long Chen, De-Chuan Zhan, Han-Jia Ye
摘要
Document parsing, the task of extracting diverse content from PDFs while preserving their structural integrity, has been significantly advanced by Multimodal Large Language Models (MLLMs). These models have achieved remarkable success, largely driven by extensive post-training on massive datasets. This paper therefore undertakes a deep analysis of the two dominant adaptation strategies, Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), prompted by a puzzling observation on the PDF-to-Markdown task: SFT makes a negligible impact, especially on parsing complex tables and formulas, while RL achieves substantial overall gains. To unravel the reasons, our systematic investigation reveals a clear and complementary division of labor: SFT primarily operates as a structure learner, biased towards mastering the low-entropy syntax of document layouts. While it learns the format of a table, it struggles to ensure the fidelity of its high-entropy cell content. Conversely, RL excels as a content refiner by optimizing a holistic reward that reflects final accuracy. We further ground this phenomenon in the distinct theoretical nature of their respective objective functions. Based on these findings, we introduce a unified strategy that explicitly harnesses their individual strengths while mitigating their weaknesses. This work shows that a deep understanding of post-training methods is key to unlocking performance beyond what data scaling alone can achieve.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper18
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida 等NeurIPS 2022 · 被引用 24,707 次
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning 等NeurIPS 2023 · 被引用 10,924 次
- Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu 等ICLR 2022 · 被引用 4,966 次
- Multitask Prompted Training Enables Zero-Shot Task GeneralizationVictor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach 等ICLR 2022 · 被引用 1,976 次
- Cross-Task Generalization via Natural Language Crowdsourcing InstructionsSwaroop Mishra, Daniel Khashabi, Chitta Baral, Hannaneh HajishirziACL 2022 · 被引用 887 次
相关 Paper
- Unveiling the Compositional Ability Gap in Vision-Language Reasoning ModelTianle Li, Jihai Zhang, Yongming Rao, Yu ChengNeurIPS 2025 · 被引用 17 次
- UFT: Unifying Supervised and Reinforcement Fine-TuningMingyang Liu, Gabriele Farina, Asuman OzdaglarNeurIPS 2025 · 被引用 61 次
- RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language ModelsYeongtak Oh, Dohyun Chung, Juhyeon Shin, Sangha Park 等NeurIPS 2025 · 被引用 12 次
- Table2LaTeX-RL: High-Fidelity LaTeX Code Generation from Table Images via Reinforced Multimodal Language ModelsJun Ling, Yao Qi, Tao Huang, Shibo Zhou 等NeurIPS 2025 · 被引用 9 次
- SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for ReasoningYuqian Fu, Tinghong Chen, Jiajun Chai, Xihuai Wang 等ICLR 2026 · 被引用 97 次
