MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe
Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Wenshuo Ma, Zhihui He, Tianchi Cai, Weize Chen, Yuxiang Huang, Ranchi Zhao, Bokai Xu, Junbo Cui
摘要
Multimodal Large Language Models (MLLMs) are undergoing rapid progress and represent the frontier of AI development. However, their training and inference efficiency have emerged as a core bottleneck in making MLLMs more accessible and scalable. To address the challenges, we present MiniCPM-V 4.5, an 8B parameter model designed for high efficiency and strong performance. We introduce three core improvements in model architecture, data strategy and training method: a unified 3D-Resampler model architecture for highly compact encoding over images and videos, a unified learning paradigm for document knowledge and text recognition without heavy data engineering, and a hybrid reinforcement learning strategy for proficiency in both short and long reasoning modes. Comprehensive experimental results in OpenCompass evaluation show that MiniCPM-V 4.5 surpasses widely used proprietary models such as GPT-4o-latest, and significantly larger open-source models such as Qwen2.5-VL 72B. Notably, the strong performance is achieved with remarkable efficiency. For example, on the widely adopted VideoMME benchmark, MiniCPM-V 4.5 achieves state-of-the-art performance among models under 30B size, using just 46.7% GPU memory cost and 8.7% inference time of Qwen2.5-VL 7B.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper26
- Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and GroundingChristopher Clark, Jieyu Zhang, Zixian Ma, Jae Sung Park 等CVPR 2026 · 被引用 144 次
- Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal PerceptionLai Wei, Liangbo He, jun lan, Lingzhong Dong 等ICML 2026 · 被引用 27 次
- Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-FollowingTianyi Xiong, Yi Ge, Ming Li, Zuolong Zhang 等CVPR 2026 · 被引用 16 次
- ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG InterpretationJiarui Jin, Haoyu Wang, Xingliang Wu, Xiaocheng Fang 等ICML 2026 · 被引用 11 次
- GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional EvaluationRang Li, Lei Li, Shuhuai Ren, Hao Tian 等CVPR 2026 · 被引用 10 次
它引用的顶会 Paper26
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning 等NeurIPS 2023 · 被引用 10,924 次
- Frozen in Time: A Joint Video and Image Encoder for End-to-End RetrievalMax Bain, Arsha Nagrani, Gül Varol, Andrew ZissermanICCV 2021 · 被引用 1,550 次
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual ContextsPan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu 等ICLR 2024 · 被引用 1,472 次
- MM-Vet: Evaluating Large Multimodal Models for Integrated CapabilitiesWeihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang 等ICML 2024 · 被引用 1,191 次
相关 Paper
- MiniGPT-3D: Efficiently Aligning 3D Point Clouds with Large Language Models using 2D PriorsYuan Tang, Xu Han, Xianzhi Li, Qiao Yu 等ACM MM 2024 · 被引用 21 次
- BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile DevicesXudong Lu, Yinghao Chen, Cheng Chen, Hui Tan 等CVPR 2025
- MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Static QuantizationJiangyong Yu, Sifan Zhou, Dawei Yang, Shuoyu Li 等ACM MM 2025 · 被引用 11 次
- LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision TokenShaolei Zhang, Qingkai Fang, Zhe Yang, Yang FengICLR 2025
- Incentivizing Versatile Video Reasoning in MLLMs via Data-Efficient Reinforcement LearningXiaodong Wang, Zhirong Wu, Langling Huang, Yuxi Zheng 等CVPR 2026
