Expressional Region Retrieval
Xiaoqian Guo, Xiangyang Li, Shuqiang Jiang
Abstract
Image retrieval is a long-standing topic in the multimedia community due to its various applications, e.g., product search and artworks retrieval in museum. The regions in images contain a wealth of information. Users may be interested in the objects presented in the image regions or the relationships between them. But previous retrieval methods are either limited to the single object of images, or tend to the entire visual scene. In this paper, we introduce a new task called expressional region retrieval, in which the query is formulated as a region of image with the associated description. The goal is to find images containing the similar content with the query and localize the regions within them. As far as we know, this task has not been explored yet. We propose a framework to address this issue. The region proposals are first generated based on region detectors and language features are extracted. Then the Gated Residual Network (GRN) takes language information as a gate to control the transformation of visual features. In this way, the combined visual and language representation is more specific and discriminative for expressional region retrieval. We evaluate our method on a new established benchmark which is constructed based on the Visual Genome dataset. Experimental results demonstrate that our model effectively utilizes both visual and language information, outperforming the baseline methods.
Ask Lune
Know where this paper stands in minutes, not an afternoon.
Lune can read the top-tier papers that cite it and the ones closest to it, and tells you what it claims, what came after and where it falls short, each point cited to the paper it comes from. No skimming, no AI slop.
Related papers
- Advancing Referring Expression Segmentation Beyond Single ImageYixuan Wu, Zhao Zhang, Chi Xie, Feng Zhu et al.ICCV 2023 · 25 citations
- PhraseCut: Language-Based Image Segmentation in the WildChenyun Wu, Zhe Lin, Scott Cohen, Trung Bui et al.CVPR 2020
- Composed Query Image Retrieval Using Locally Bounded FeaturesMehrdad Hosseinzadeh, Yang WangCVPR 2020
- Referring Expression Instance Retrieval and A Strong End-to-End BaselineXiangzhao Hao, Kuan Zhu, Hongyu Guo, Haiyun Guo et al.ACM MM 2025
- Visual Grounding in Remote Sensing ImagesYuxi Sun, Shanshan Feng, Xutao Li, Yunming Ye et al.ACM MM 2022 · 76 citations
