CVPR Findings2026
OKGraph: Online Knowledge Graph Probing for Open-vocabulary Recognition
Junhui Yin, Zhizhen Cai, Puze Wang, Guanzhou Ke, Jianhua Yang, Man Zhang, Qiang Zhang, Shengfeng He
Abstract
Vision-language models (VLMs) have made significant strides in open-vocabulary recognition by aligning visual inputs with textual representations. However, most existing approaches treat categories as independent labels and rely on prompt tuning or feature adaptation, overlooking the underlying semantic structure that links related concepts. This lack of semantic connectivity limits the models' ability to generalize, especially when the data distribution shifts or categories exhibit semantic ambiguity, where recognizing instances often depends on understanding their relationships to known concepts. We propose Online Knowledge Graph Probing (OKGraph), a dynamic framework that constructs, updates, and applies a semantic knowledge graph to guide recognition. OKGraph integrates the perceptual capabilities of VLMs with the semantic reasoning of large language models (LLMs), enabling the adaptive extraction of contextual knowledge from test data. Through iterative contextual probing and model-guided correction, OKGraph refines its knowledge graph without requiring any training data or manual annotations. Extensive experiments across diverse vision tasks and domains demonstrate that OKGraph substantially improves zero-shot and few-shot performance, highlighting the value of structured reasoning in open-world vision. The code is available at https://github.com/zenithc-git/OKGraph.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation cites Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation cites Learning Transferable Visual Models From Natural Language Supervision
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision cites The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision cites Learning Transferable Visual Models From Natural Language Supervision
- Learning Transferable Visual Models From Natural Language Supervision cites The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
- Flamingo: a Visual Language Model for Few-Shot Learning cites BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
- Flamingo: a Visual Language Model for Few-Shot Learning cites Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
- Flamingo: a Visual Language Model for Few-Shot Learning cites Learning Transferable Visual Models From Natural Language Supervision
Builds on24
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 citations
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text SupervisionChao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen et al.ICML 2021 · 5,401 citations
- The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution GeneralizationDan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath et al.ICCV 2021 · 2,294 citations
Related papers
- Efficient and Context-Aware Label Propagation for Zero-/Few-Shot Training-Free Adaptation of Vision-Language ModelYushu Li, Yongyi Su, Adam Goodge, Kui Jia et al.ICLR 2025
- LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image CollectionsMuhammad Jehanzeb Mirza, Leonid Karlinsky, Wei Lin, Horst Possegger et al.NeurIPS 2023 · 63 citations
- View-on-Graph: Zero-Shot 3D Visual Grounding via Vision-Language Reasoning on Scene GraphsYuanyuan Liu, Haiyang Mei, Dongyang Zhan, Jiayue Zhao et al.AAAI 2026 · 1 citation
- Tree of Attributes Prompt Learning for Vision-Language ModelsTong Ding, Wanhua Li, Zhongqi Miao, Hanspeter PfisterICLR 2025
- Open-World 3D Scene Graph Generation for Retrieval-Augmented ReasoningFei Yu, Quan Deng, Shengeng Tang, Yuehua Li et al.AAAI 2026 · 2 citations
