Lune

CVPR Findings2026

KGGAT: Knowledge-Guided Graph Attention Network for Multi-Label Image Classification

Christine Dewi, Dhananjay R Thiruvady, Nayyar Zaidi

Abstract

Multi-label image classification requires understanding not only the visual features of individual objects but also their semantic relationships within a scene. To address this challenge, we propose KGGAT -[K]nowledge-[G]uided [G]raph [At]tention Network that integrates external knowledge graphs with attention-based visual reasoning. Unlike conventional transformer-based models that rely solely on visual correlations, KGGAT incorporates structured semantic priors from ConceptNet55 and OpenAI-derived knowledge graphs to explicitly model inter-label dependencies. A dual-path architecture couples a Graph Attention Network (GAT) for label reasoning with a transformerbased visual encoder, enabling cross-modal alignment between visual and semantic spaces. In addition, an elementwise addition fusion strategy and an Asymmetric Loss (ASL) objective enhance feature interaction and effectively mitigate class imbalance. Extensive experiments on MS-COCO and VOC 2007 demonstrate that KGGAT achieves state-of-the-art performance, attaining 89.21% mAP and 96.85% mAP, respectively. Qualitative analyses further show that KGGAT provides accurate object localization and interpretable label reasoning across diverse scenes, validating its effectiveness in knowledgeguided multi-label image classification. Code and pretrained models will be released to facilitate future research on knowledge-guided visual understanding.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Asymmetric Loss For Multi-Label Classification cites Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition
  • Asymmetric Loss For Multi-Label Classification cites Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification
  • Residual Attention: A Simple but Effective Method for Multi-Label Recognition cites Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition
  • Residual Attention: A Simple but Effective Method for Multi-Label Recognition cites Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification
  • Residual Attention: A Simple but Effective Method for Multi-Label Recognition cites Asymmetric Loss For Multi-Label Classification
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on7

  1. Asymmetric Loss For Multi-Label ClassificationTal Ridnik, Emanuel Ben Baruch, Nadav Zamir, Asaf Noy et al.ICCV 2021 · 778 citations
  2. Learning Semantic-Specific Graph Representation for Multi-Label Image RecognitionTianshui Chen, Muxin Xu, Xiaolu Hui, Hefeng Wu et al.ICCV 2019 · 347 citations
  3. Cross-Modality Attention with Semantic Graph Embedding for Multi-Label ClassificationRenchun You, Zhiyao Guo, Lei Cui, Xiang Long et al.AAAI 2020 · 221 citations
  4. Residual Attention: A Simple but Effective Method for Multi-Label RecognitionKe Zhu, Jianxin WuICCV 2021 · 190 citations
  5. Collaborative Transformers for Grounded Situation RecognitionJunhyeong Cho, Youngseok Yoon, Suha KwakCVPR 2022 · 23 citations

Related papers

  1. Simple Yet Effective: Structure Guided Pre-trained Transformer for Multi-modal Knowledge Graph ReasoningKe Liang, Lingyuan Meng, Yue Liu, Meng Liu et al.ACM MM 2024 · 29 citations
  2. Explicit Knowledge Incorporation for Visual ReasoningYifeng Zhang, Ming Jiang, Qi ZhaoCVPR 2021
  3. Multimodal Reasoning with Multimodal Knowledge GraphJunlin Lee, Yequan Wang, Jing Li, Min ZhangACL 2024 · 29 citations
  4. Relation-Aware Language-Graph Transformer for Question AnsweringJinyoung Park, Hyeong Kyu Choi, Juyeon Ko, Hyeon-Jin Park et al.AAAI 2023 · 16 citations
  5. Multi-Granularity Multi-Modal Knowledge Graph Representation Learning via Subgraph-Aware Adaptive Fusion and Hierarchical Relation ModelingPeining Li, Meiyu Liang, Wei Huang, Junping Du et al.WWW 2026
Dusk over the sea between two cliffs drawn in fine vertical lines