GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection
Jiaming Li, Zhijia Liang, Weikai Chen, Lin Ma, Guanbin Li
Abstract
Fine-grained open-vocabulary object detection (FG-OVD) aims to detect novel object categories described by attribute-rich texts. While existing open-vocabulary detectors show promise at the base-category level, they underperform in fine-grained settings due to the semantic entanglement of subjects and attributes in pretrained vision-language model (VLM) embeddings -- leading to over-representation of attributes, mislocalization, and semantic drift in embedding space. We propose GUIDED, a decomposition framework specifically designed to address the semantic entanglement between subjects and attributes in fine-grained prompts. By separating object localization and fine-grained recognition into distinct pathways, HUIDED aligns each subtask with the module best suited for its respective roles. Specifically, given a fine-grained class name, we first use a language model to extract a coarse-grained subject and its descriptive attributes. Then the detector is guided solely by the subject embedding, ensuring stable localization unaffected by irrelevant or overrepresented attributes. To selectively retain helpful attributes, we introduce an attribute embedding fusion module that incorporates attribute information into detection queries in an attention-based manner. This mitigates over-representation while preserving discriminative power. Finally, a region-level attribute discrimination module compares each detected region against full fine-grained class names using a refined vision-language model with a projection head for improved alignment. Extensive experiments on FG-OVD and 3F-OVD benchmarks show that GUIDED achieves new state-of-the-art results, demonstrating the benefits of disentangled modeling and modular optimization. Our code will be released at https://github.com/lijm48/GUIDED.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers4
- Cross-Modal Attention Calibration for LVLM Hallucination MitigationJiaming Li, Jiacheng Zhang, Zequn Jie, Lin Ma et al.CVPR 2026 · 23 citations
- OASIS: On-Demand Hierarchical Event Memory for Streaming Video ReasoningZhijia Liang, Jiaming Li, Weikai Chen, Yanhao Zhang et al.CVPR 2026 · 16 citations
- SRA-Det: Learning Omni-Grained Open-Vocabulary Detection Beyond Category NamesLi Yang, Boyu Cai, Wei Liu, Yan Wang et al.CVPR 2026 · 1 citation
- ObjEmbed: Towards Universal Multimodal Object EmbeddingsShenghao Fu, Yukun Su, Fengyun Rao, Jing LYU et al.ICML 2026
Builds on25
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Emerging Properties in Self-Supervised Vision TransformersMathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou et al.ICCV 2021 · 8,921 citations
- A ConvNet for the 2020sZhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer et al.CVPR 2022 · 6,782 citations
- Open-vocabulary Object Detection via Vision and Language Knowledge DistillationXiuye Gu, Tsung-Yi Lin, Weicheng Kuo, Yin CuiICLR 2022 · 1,274 citations
- MDETR - Modulated Detection for End-to-End Multi-Modal UnderstandingAishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve et al.ICCV 2021 · 1,114 citations
Related papers
- Language-driven Fine-grained RetrievalShijie Wang, Xin Yu, Yadan Luo, Zijian Wang et al.CVPR 2026 · 2 citations
- Open-Vocabulary Semantic Segmentation via Attribute Decomposition-AggregationChaofan Ma, Yuhuan Yang, Chen Ju, Fei Zhang et al.NeurIPS 2023 · 40 citations
- LLMs Meet VLMs: Boost Open Vocabulary Object Detection with Fine-grained DescriptorsSheng Jin, Xueying Jiang, Jiaxing Huang, Lewei Lu et al.ICLR 2024 · 48 citations
- Addressing Granularity-induced Semantic Drift in OvOD via Graph-guided semantically consistent representationHongyan Xu, Zhongze Wu, Ang He, Xi Lin et al.ACM MM 2025
- The Devil is in the Fine-Grained Details: Evaluating open-Vocabulary Object Detectors for Fine-Grained UnderstandingLorenzo Bianchi, Fabio Carrara, Nicola Messina, Claudio Gennaro et al.CVPR 2024
