EMNLP Findings2025
BcQLM: Efficient Vision-Language Understanding with Distilled Q-Gated Cross-Modal Fusion
摘要
As multimodal large language models (MLLMs) advance, their large-scale architectures pose challenges for deployment in resource-constrained environments. In the age of large models, where energy efficiency, computational scalability and environmental sustainability are paramount, the development of lightweight and high-performance models is critical for real-world applications. As such, we propose a lightweight MLLM framework for end-to-end visual question answering. Our proposed approach centres on BreezeCLIP, a compact yet powerful vision-language encoder optimised for efficient multimodal understanding. With only 1.2 billion parameters overall, our model significantly reduces computational cost while achieving performance comparable to standard-size MLLMs. Experiments conducted on multiple datasets further validate its effectiveness in balancing accuracy and efficiency. The modular and extensible design enables generalisation to broader multimodal tasks. The proposed lightweight vision-language framework is denoted as BcQLM (BreezeCLIPenhanced Q-Gated Multimodal Language Model). It offers a promising path toward deployable MLLMs under practical hardware constraints. The source code is available at https://github.com/thico0224/BcQLM.
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
脉络
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning 引用了 LoRA: Low-Rank Adaptation of Large Language Models
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning 引用了 BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning 引用了 BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation 引用了 Learning Transferable Visual Models From Natural Language Supervision
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models 引用了 BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
它引用的 Paper15
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 47,906 次引用
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 18,833 次引用
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 次引用
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 次引用
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningWenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong 等NeurIPS 2023 · 4,013 次引用
相关 Paper
- Eve: Efficient Multimodal Vision Language Models with Elastic Visual ExpertsMiao Rang, Zhenni Bi, Chuanjian Liu, Yehui Tang 等AAAI 2025 · 16 次引用
- Q-CLIP: Unleashing the Power of Vision-Language Models for Video Quality Assessment through Unified Cross-Modal AdaptationYachun Mi, Yu Li, Yanting Li, Chen Hui 等ICML 2026
- DocSLM: A Small Vision-Language Model for Long Multimodal Document UnderstandingTanveer Hannan, Dimitrios Mallios, Parth Pathak, Faegheh Sardari, Thomas Seidl, Gedas Bertasius, Mohsen Fayyaz, Sunando SenguptaCVPR Findings 2026
- Seeing Is Coding: On the Effectiveness of Vision Language Models in Code UnderstandingYuling Shi, Chaoxiang Xie, Zhensu Sun, Yeheng Chen 等ISSTA 2026 · 1 次引用
- BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile DevicesXudong Lu, Yinghao Chen, Cheng Chen, Hui Tan 等CVPR 2025
