SRA-Det: Learning Omni-Grained Open-Vocabulary Detection Beyond Category Names
Li Yang, Boyu Cai, Wei Liu, Yan Wang, Chunfeng Yuan, Bing Li, Weiming Hu
Abstract
Open-vocabulary object detection (OVD) aims to detect objects described by arbitrary text, but most existing methods operate at a coarse category level and struggle with fine-grained, attribute-sensitive queries. We address this from both model and data perspectives. We propose a Semantic-Retrieval-Augmented Detector (SRA-Det) that uses an attention-based module to retrieve multiple semantic facets from token-level text features, and a soft-min matching rule that behaves like a differentiable logical AND over these facets, ensuring that all key attributes are satisfied. In parallel, we introduce an automatic attributeaugmented data pipeline that uses an LLM to generate category-specific visual attributes and a dual CLIP-based similarity check to verify them at the instance level. With a Swin-T backbone, our approach achieves 54.9 mAP in the zero-shot setting on FG-OVD and 40.4 AP on LVIS, establishing strong fine-grained and general OVD performance.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on18
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsZe Liu, Yutong Lin, Yue Cao, Han Hu et al.ICCV 2021 · 31,683 citations
- Open-vocabulary Object Detection via Vision and Language Knowledge DistillationXiuye Gu, Tsung-Yi Lin, Weicheng Kuo, Yin CuiICLR 2022 · 1,274 citations
- Objects365: A Large-Scale, High-Quality Dataset for Object DetectionShuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng et al.ICCV 2019 · 1,018 citations
- Scaling Open-Vocabulary Object DetectionMatthias Minderer, Alexey A. Gritsenko, Neil HoulsbyNeurIPS 2023 · 482 citations
Related papers
- DetCLIPv3: Towards Versatile Generative Open-Vocabulary Object DetectionLewei Yao, Renjie Pi, Jianhua Han, Xiaodan Liang et al.CVPR 2024 · 22 citations
- GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object DetectionJiaming Li, Zhijia Liang, Weikai Chen, Lin Ma et al.NeurIPS 2025 · 6 citations
- Multi-Modal Classifiers for Open-Vocabulary Object DetectionPrannay Kaul, Weidi Xie, Andrew ZissermanICML 2023 · 69 citations
- OvarNet: Towards Open-Vocabulary Object Attribute RecognitionKeyan Chen, Xiaolong Jiang, Yao Hu, Xu Tang et al.CVPR 2023
- CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-MatchingXiaoshi Wu, Feng Zhu, Rui Zhao, Hongsheng LiCVPR 2023
