Synergistic Space-Vision Processing for Predicate Inference
Zhenhua Lei, Zefang Han, yu qiu
Abstract
Scene graph generation (SGG) aims to parse an image into a structured graph of objects and their predicates, enabling explicit relational reasoning for visual understanding. However, prevailing methods often over-predict geometric predicates, resulting in scene graphs that are factually correct yet semantically shallow. While recent works effectively attribute this phenomenon to the long-tailed data distribution, we identify another critical factor driving such biased prediction: co-occurrence-induced representation entanglement, where geometric and non-geometric predicates that frequently co-occur are encoded into overly similar representations. To this end, we introduce Dual-stream Synergistic Network (DS-Net) that models geometric and non-geometric predicates with two specialized streams, coupled with a bidirectional cross-stream fusion mechanism. The space stream focuses on spatial and structural cues, while the vision stream captures fine-grained visual evidence and semantic priors. Extensive experiments show that DS-Net consistently improves predicate inference, achieving 1.3% 6.1% absolute gains in mR@100 on the SGGen task when integrated into existing SGG methods. These results highlight the importance of synergistic modeling of geometric and non-geometric predicates for generating semantically richer scene graphs.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on23
- NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving ScenarioTianwen Qian, Jingjing Chen, Linhai Zhuo, Yang Jiao et al.AAAI 2024 · 314 citations
- Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph GenerationXingning Dong, Tian Gan, Xuemeng Song, Jianlong Wu et al.CVPR 2022 · 116 citations
- Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question AnsweringWeizhe Lin, Jinghong Chen, Jingbiao Mei, Alexandru Coca et al.NeurIPS 2023 · 108 citations
- The Devil is in the Labels: Noisy Label Correction for Robust Scene Graph GenerationLin Li, Long Chen, Yifeng Huang, Zhimeng Zhang et al.CVPR 2022 · 103 citations
- Context-aware Scene Graph Generation with Seq2Seq TransformersYichao Lu, Himanshu Rai, Jason Chang, Boris Knyazev et al.ICCV 2021 · 93 citations
Related papers
- Leveraging Predicate and Triplet Learning for Scene Graph GenerationJiankai Li, Yunhong Wang, Xiefan Guo, Ruijie Yang et al.CVPR 2024
- UniQ: Unified Decoder with Task-specific Queries for Efficient Scene Graph GenerationXinyao Liao, Wei Wei, Dangyang Chen, Yuanyuan FuACM MM 2024 · 2 citations
- Learning to Generate an Unbiased Scene Graph by Using Attribute-Guided Predicate FeaturesLei Wang, Zejian Yuan, Badong ChenAAAI 2023 · 8 citations
- Weakly Supervised Visual Semantic ParsingAlireza Zareian, Svebor Karaman, Shih-Fu ChangCVPR 2020
- End-to-End Entity-Predicate Association Reasoning for Dynamic Scene Graph GenerationLiwei Wang, Yanduo Zhang, Tao Lu, Fang Liu et al.ICCV 2025 · 1 citation
