Lune

ICCV2023Top-tier venue

Prototype-based Dataset Comparison

Nanne van Noord

Abstract

Dataset summarisation is a fruitful approach to dataset inspection. However, when applied to a single dataset the discovery of visual concepts is restricted to those most prominent. We argue that a comparative approach can expand upon this paradigm to enable richer forms of dataset inspection that go beyond the most prominent concepts.To enable dataset comparison we present a module that learns concept-level prototypes across datasets. We leverage self-supervised learning to discover these prototypes without supervision, and we demonstrate the benefits of our approach in two case-studies. Our findings show that dataset comparison extends dataset inspection and we hope to encourage more works in this direction. Code and usage instructions available at https://github.com/Nanne/ProtoSim

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

6 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Emerging Properties in Self-Supervised Vision Transformers cites A Simple Framework for Contrastive Learning of Visual Representations
  • Emerging Properties in Self-Supervised Vision Transformers cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Emerging Properties in Self-Supervised Vision Transformers cites Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning
  • Emerging Properties in Self-Supervised Vision Transformers cites Training data-efficient image transformers & distillation through attention
  • ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts cites Describing Differences in Image Sets with Natural Language
  • ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale cites A Simple Framework for Contrastive Learning of Visual Representations
  • Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning cites A Simple Framework for Contrastive Learning of Visual Representations
  • Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections of Images cites Describing Differences in Image Sets with Natural Language
  • Training data-efficient image transformers & distillation through attention cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge cites Emerging Properties in Self-Supervised Vision Transformers
  • Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • ImageSet2Text: Describing Sets of Images Through Text cites Describing Differences in Image Sets with Natural Language
  • ImageSet2Text: Describing Sets of Images Through Text cites Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections of Images
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers6

  1. Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual KnowledgeFawaz Sammani, Nikos DeligiannisNeurIPS 2024 · 15 citations
  2. ConceptScope: Characterizing Dataset Bias via Disentangled Visual ConceptsJinho Choi, Hyesu Lim, Steffen Schneider, Jaegul ChooNeurIPS 2025 · 5 citations
  3. Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections of ImagesBoyang Deng, Songyou Peng, Kyle Genova, Gordon Wetzstein et al.ICCV 2025 · 4 citations
  4. ImageSet2Text: Describing Sets of Images Through TextPiera Riccio, Francesco Galati, Kajetan Schweighofer, Noa Garcia et al.AAAI 2026 · 1 citation
  5. Describing Differences in Image Sets with Natural LanguageLisa Dunlap, Yuhui Zhang, Xiaohan Wang, Ruiqi Zhong et al.CVPR 2024

Builds on17

  1. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  2. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  3. Bootstrap Your Own Latent - A New Approach to Self-Supervised LearningJean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec et al.NeurIPS 2020 · 9,171 citations
  4. Training data-efficient image transformers & distillation through attentionHugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa et al.ICML 2021 · 8,974 citations
  5. Emerging Properties in Self-Supervised Vision TransformersMathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou et al.ICCV 2021 · 8,921 citations

Related papers

  1. CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud UnderstandingMohamed Afham, Isuru Dissanayake, Dinithi Dissanayake, Amaya Dharmasiri et al.CVPR 2022 · 286 citations
  2. Dataset Ownership Verification in Contrastive Pre-trained ModelsYuechen Xie, Jie Song, Mengqi Xue, Haofei Zhang et al.ICLR 2025
  3. Coreset Sampling from Open-Set for Fine-Grained Self-Supervised LearningSungnyun Kim, Sangmin Bae, Se-Young YunCVPR 2023
  4. When Does Contrastive Visual Representation Learning Work?Elijah Cole, Xuan Yang, Kimberly Wilber, Oisin Mac Aodha et al.CVPR 2022 · 98 citations
  5. Prototypical Cross-domain Knowledge Transfer for Cervical Dysplasia Visual InspectionYichen Zhang, Yifang Yin, Ying Zhang, Zhenguang Liu et al.ACM MM 2023 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines