DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understanding
Abstract
Humans can effortlessly locate desired objects in cluttered environments, relying on a cognitive mechanism known as visual search to efficiently filter out irrelevant information and focus on task-related regions. Inspired by this process, we propose DyFo (Dynamic Focus), a training-free dynamic focusing visual search method that enhances fine-grained visual understanding in large multimodal models (LMMs). Unlike existing approaches which require additional modules or data collection, DyFo leverages a bidirectional interaction between LMMs and visual experts, using a Monte Carlo Tree Search (MCTS) algorithm to simulate human-like focus adjustments. This enables LMMs to focus on key visual regions while filtering out irrelevant content, without introducing additional training caused by vocabulary expansion or the integration of specialized localization modules. Experimental results demonstrate that DyFo significantly improves fine-grained visual understanding and reduces hallucination issues in LMMs, achieving superior performance across both fixed and dynamic resolution models.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
30 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Look-Back: Implicit Visual Re-focusing in MLLM Reasoning cites Visual Instruction Tuning
- Thinking in 360deg: Humanoid Visual Search in the Wild cites Visual Instruction Tuning
- Thinking in 360deg: Humanoid Visual Search in the Wild cites Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search
- Thinking in 360deg: Humanoid Visual Search in the Wild cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- Thinking in 360deg: Humanoid Visual Search in the Wild cites Flamingo: a Visual Language Model for Few-Shot Learning
- Thinking in 360deg: Humanoid Visual Search in the Wild cites DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning
- Visual Instruction Tuning cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
- Visual Instruction Tuning cites Flamingo: a Visual Language Model for Few-Shot Learning
- Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search cites Visual Instruction Tuning
- Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search cites Flamingo: a Visual Language Model for Few-Shot Learning
- Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search cites DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models cites Flamingo: a Visual Language Model for Few-Shot Learning
- Learning Transferable Visual Models From Natural Language Supervision cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Flamingo: a Visual Language Model for Few-Shot Learning cites Learning Transferable Visual Models From Natural Language Supervision
- Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning cites DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning cites Visual Instruction Tuning
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning cites Flamingo: a Visual Language Model for Few-Shot Learning
Cited by top-tier papers30
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement LearningZiwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao et al.ICLR 2026 · 321 citations
- Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual SearchXin Lai, Junyi Li, Wei Li, Tao Liu et al.ICLR 2026 · 124 citations
- Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video ReasoningHaoji Zhang, Xin Gu, Jiawen Li, Chixiang Ma et al.CVPR 2026 · 92 citations
- Look-Back: Implicit Visual Re-focusing in MLLM ReasoningShuo Yang, Yuwei Niu, Yuyang Liu, Yang Ye et al.AAAI 2026 · 32 citations
- Thinking in 360deg: Humanoid Visual Search in the WildHeyang Yu, Yinan Han, Xiangyu Zhang, Baiqiao Yin et al.CVPR 2026 · 16 citations
Builds on21
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
- Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations
Related papers
- FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question AnsweringLiangyu Zhong, Fabio Rosenthal, Joachim Sicking, Fabian Hüger et al.NeurIPS 2025 · 23 citations
- Seeing What Matters: A Training-Free Self-Guided Framework for Multimodal Detail Perception and ReasoningMingjie Ma, yichao ma, Zhong Yang, Guohui LiCVPR 2026
- Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language ModelsJianghao Yin, Qin Chen, Kedi Chen, Jie Zhou et al.ICLR 2026 · 7 citations
- ActiveScope: Actively Seeking and Correcting Perception for MLLMsYajing Wang, Chao Bi, Junshu Sun, Shufan Shen et al.ICML 2026 · 1 citation
- DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual ReasoningHang Wu, Hongkai Chen, Yujun Cai, Chang Liu et al.EMNLP 2025 · 23 citations
