Lune

CVPR2020Top-tier venue

PhraseCut: Language-Based Image Segmentation in the Wild

Chenyun Wu, Zhe Lin, Scott Cohen, Trung Bui, Subhransu Maji

Abstract

We consider the problem of segmenting image regions given a natural language phrase, and study it on a novel dataset of 77,262 images and 345,486 phrase-region pairs. Our dataset is collected on top of the Visual Genome dataset and uses the existing annotations to generate a challenging set of referring phrases for which the corresponding regions are manually annotated. Phrases in our dataset correspond to multiple regions and describe a large number of object and stuff categories as well as their attributes such as color, shape, parts, and relationships with other entities in the image. Our experiments show that the scale and diversity of concepts in our dataset poses significant challenges to the existing state-of-the-art. We systematically handle the long-tail nature of these concepts and present a modular approach to combine category, attribute, and relationship cues that outperforms existing approaches.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

59 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • MDETR - Modulated Detection for End-to-End Multi-Modal Understanding cites See-Through-Text Grouping for Referring Image Segmentation
  • MDETR - Modulated Detection for End-to-End Multi-Modal Understanding cites A Fast and Accurate One-Stage Approach to Visual Grounding
  • GLIPv2: Unifying Localization and Vision-Language Understanding cites MDETR - Modulated Detection for End-to-End Multi-Modal Understanding
  • Image Segmentation Using Text and Image Prompts cites MDETR - Modulated Detection for End-to-End Multi-Modal Understanding
  • SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentation cites Image Segmentation Using Text and Image Prompts
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers59

  1. MDETR - Modulated Detection for End-to-End Multi-Modal UnderstandingAishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve et al.ICCV 2021 · 1,114 citations
  2. Image Segmentation Using Text and Image PromptsTimo Lüddecke, Alexander S. EckerCVPR 2022 · 457 citations
  3. GLIPv2: Unifying Localization and Vision-Language UnderstandingHaotian Zhang, Pengchuan Zhang, Xiaowei Hu, Yen-Chun Chen et al.NeurIPS 2022 · 403 citations
  4. MeViS: A Large-scale Benchmark for Video Segmentation with Motion ExpressionsHenghui Ding, Chang Liu, Shuting He, Xudong Jiang et al.ICCV 2023 · 242 citations
  5. SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic SegmentationHuaishao Luo, Junwei Bao, Youzheng Wu, Xiaodong He et al.ICML 2023 · 222 citations

Builds on6

  1. A Fast and Accurate One-Stage Approach to Visual GroundingZhengyuan Yang, Boqing Gong, Liwei Wang, Wenbing Huang et al.ICCV 2019 · 441 citations
  2. Learning to Assemble Neural Module Tree Networks for Visual GroundingDaqing Liu, Hanwang Zhang, Feng Wu, Zheng-Jun ZhaICCV 2019 · 317 citations
  3. Dynamic Graph Attention for Referring Expression ComprehensionSibei Yang, Guanbin Li, Yizhou YuICCV 2019 · 251 citations
  4. Zero-Shot Grounding of Objects From Natural Language QueriesArka Sadhu, Kan Chen, Ram NevatiaICCV 2019 · 176 citations
  5. See-Through-Text Grouping for Referring Image SegmentationDing-Jie Chen, Songhao Jia, Yi-Chen Lo, Hwann-Tzong Chen et al.ICCV 2019 · 153 citations

Related papers

  1. Locate Then Segment: A Strong Pipeline for Referring Image SegmentationYa Jing, Tao Kong, Wei Wang, Liang Wang et al.CVPR 2021
  2. Expressional Region RetrievalXiaoqian Guo, Xiangyang Li, Shuqiang JiangACM MM 2020 · 1 citation
  3. GLaMM: Pixel Grounding Large Multimodal ModelHanoona Abdul Rasheed, Muhammad Maaz, Sahal Shaji Mullappilly, Abdelrahman M. Shaker et al.CVPR 2024 · 113 citations
  4. 3D-DRES: Detailed 3D Referring Expression SegmentationQi Chen, Changli Wu, Jiayi Ji, Yiwei Ma et al.AAAI 2026 · 1 citation
  5. Beyond One-to-One: Rethinking the Referring Image SegmentationYutao Hu, Qixiong Wang, Wenqi Shao, Enze Xie et al.ICCV 2023 · 88 citations
Dusk over the sea between two cliffs drawn in fine vertical lines