Lune

NeurIPS2024顶会

Hierarchy-Agnostic Unsupervised Segmentation: Parsing Semantic Image Structure

Simone Rossetti, Fiora Pirri

摘要

Unsupervised semantic segmentation aims to discover groupings within images, capturing objects' view-invariance without external supervision. Moreover, this task is inherently ambiguous due to the varying levels of semantic granularity. Existing methods often bypass this ambiguity using dataset-specific priors. In our research, we address this ambiguity head-on and provide a universal tool for pixellevel semantic parsing of images guided by the latent representations encoded in self-supervised models. We introduce a novel algebraic approach that recursively decomposes an image into nested subgraphs, dynamically estimating their count and ensuring clear separation. The innovative approach identifies scene-specific primitives and constructs a hierarchy-agnostic tree of semantic regions from the image pixels. The model captures fine and coarse semantic details, producing a nuanced and unbiased segmentation. We present a new metric for estimating the quality of the semantic segmentation of discovered elements on different levels of the hierarchy. The metric validates the intrinsic nature of the compositional relations among parts, objects, and scenes in a hierarchy-agnostic domain. Our results prove the power of this methodology, uncovering semantic regions without prior definitions and scaling effectively across various datasets. This robust framework for unsupervised image segmentation proves more accurate semantic hierarchical relationships between scene elements than traditional algorithms. The experiments underscore its potential for broad applicability in image analysis tasks, showcasing its ability to deliver a detailed and unbiased segmentation that surpasses existing unsupervised methods. Related works Self-supervised representation learning for unsupervised segmentation. Self-supervised learning (SSL) is about learning representations of real-world data without human supervision [14, 36, 11,

问 Lune

几分钟读透这篇 Paper,不用再花一下午。

Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。

可以从这些问题问起

Lune 可以查阅

全文 · 相关的 Paper

获取答案

免费开始,无需绑卡

脉络

  • Emerging Properties in Self-Supervised Vision Transformers 引用了 A Simple Framework for Contrastive Learning of Visual Representations
  • Emerging Properties in Self-Supervised Vision Transformers 引用了 An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Segment Anything 引用了 An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Segment Anything 引用了 Learning Transferable Visual Models From Natural Language Supervision
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale 引用了 A Simple Framework for Contrastive Learning of Visual Representations
  • Learning Transferable Visual Models From Natural Language Supervision 引用了 A Simple Framework for Contrastive Learning of Visual Representations
  • Learning Transferable Visual Models From Natural Language Supervision 引用了 An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
它引用的引用它的从左到右按年份排列,引用越多,位置越高。

它引用的 Paper33

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 47,906 次引用
  2. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 次引用
  3. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 21,477 次引用
  4. Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao 等ICCV 2023 · 13,211 次引用
  5. Emerging Properties in Self-Supervised Vision TransformersMathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou 等ICCV 2021 · 8,921 次引用

相关 Paper

  1. Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering TransformersTsung-Wei Ke, Jyh-Jing Hwang, Yunhui Guo, Xudong Wang 等CVPR 2022 · 34 次引用
  2. Unsupervised Part Discovery from Contrastive ReconstructionSubhabrata Choudhury, Iro Laina, Christian Rupprecht, Andrea VedaldiNeurIPS 2021 · 74 次引用
  3. SmooSeg: Smoothness Prior for Unsupervised Semantic SegmentationMengcheng Lan, Xinjiang Wang, Yiping Ke, Jiaxing Xu 等NeurIPS 2023 · 28 次引用
  4. SegSort: Segmentation by Discriminative Sorting of SegmentsJyh-Jing Hwang, Stella X. Yu, Jianbo Shi, Maxwell D. Collins 等ICCV 2019 · 160 次引用
  5. Segment Anything without SupervisionXudong Wang, Jingfeng Yang, Trevor DarrellNeurIPS 2024 · 36 次引用
黄昏的海面,两侧是细线勾勒的悬崖