HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling
Xianjie Liu, Yiman Hu, Yixiong Zou, Liang Wu, Jian Xu, Bo Zheng
摘要
Multimodal Large Language Models have made substantial progress on visual understanding tasks, yet they still perform poorly on high-resolution images. Prior work often attributes this limitation to perceptual constraints, arguing that MLLMs fail to recognize small objects and therefore rely on ''zoom-in" strategies to recover fine details. In contrast, our analysis shows that the dominant failure mode is background interference rather than object size. We study the "zoom-in" operation through a hierarchical decoupling analysis and propose the Hierarchical Decoupling Framework , a training-free method that turns implicit attention into explicit region selection. HiDe first performs Token-wise Attention Decoupling to disentangle question semantics and identify the most informative tokens, then uses their attention patterns to pinpoint the corresponding visual regions. It subsequently applies Layout-Preserving Decoupling to extract these regions from cluttered backgrounds and construct a compact representation that retains key spatial structure while filtering out irrelevant context. HiDe achieves state-of-the-art results on high-resolution benchmarks like Vstar Bench. It boosts Qwen2.5-VL 7B and InternVL3 8B to state of the art performance, reaching 92.1% and 91.6% on Vstar Bench, and even surpasses reinforcement learning based methods. After optimization, HiDe reduces memory usage by 75% compared with the previous training-free approach. Code will be available at https://tennine2077.github.io/HiDe.github.io/.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper6
- Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal PerceptionLai Wei, Liangbo He, jun lan, Lingzhong Dong 等ICML 2026 · 被引用 27 次
- What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-ZoomMa Yan, Weiyu Zhang, Tianle Li, Du Linge 等ICML 2026 · 被引用 5 次
- MRD: Multi-resolution Retrieval-Detection Fusion for High-Resolution Image UnderstandingFan Yang, Xingping Dong, Xin Yu, Wenhan Luo 等CVPR 2026 · 被引用 1 次
- ActiveScope: Actively Seeking and Correcting Perception for MLLMsYajing Wang, Chao Bi, Junshu Sun, Shufan Shen 等ICML 2026 · 被引用 1 次
- Multimodal Nested Learning for Decoupled and Coordinated OptimizationYanglin Feng, Yang Qin, Dezhong Peng, Rui Wang 等ICML 2026
它引用的顶会 Paper19
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 被引用 21,477 次
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 被引用 11,349 次
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 被引用 7,873 次
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessTri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra 等NeurIPS 2022 · 被引用 5,493 次
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang 等NeurIPS 2025 · 被引用 1,109 次
相关 Paper
- FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question AnsweringLiangyu Zhong, Fabio Rosenthal, Joachim Sicking, Fabian Hüger 等NeurIPS 2025 · 被引用 23 次
- Seeing What Matters: A Training-Free Self-Guided Framework for Multimodal Detail Perception and ReasoningMingjie Ma, yichao ma, Zhong Yang, Guohui LiCVPR 2026
- Hi-Lo Prune: Look at What You'll Lose before Pruning with Hierarchical Token SelectionZixun Sun, Yubo Dong, Hehe Fan, Yi YangCVPR 2026
- Deeper Thought, Weaker Aim: Understanding and Mitigating Perceptual Impairment during Reasoning in Multimodal Large Language ModelsRuiying Peng, Xueyu Wu, Jing Lei, Lu Hou 等CVPR 2026 · 被引用 4 次
- MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMsJiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, Filip IlievskiICLR 2025 · 被引用 1 次
