Adapt Foundational Segmentation Models with Heterogeneous Searching Space
Li Yi, Jie Hu, Songan Zhang, Guannan Jiang
Abstract
To boost segmentation performance while keeping zero-shot features, one approach is pre-augmenting images for the segmentation model. However, existing image augmentations mainly depend on rule-based methods, restricting augmentation effectiveness. Though learningbased methods can diversify augmentation, rule-based ones are degree-describable (e.g., slight/intense brightening), while learning-based methods usually predict non-degreedescribable ground truths (e.g., depth estimation), creating a heterogeneous search space when combined. To this end, we propose an "Augmenting-to-Adapt" paradigm, replacing traditional rule-based augmentation with an optimal heterogeneous augmentation policy to enhance segmentation. Our method uses 32 augmentation techniques (22 rule-based, 10 learning-based) to ease parameter misalignment, forming a robust, multi-discrete heterogeneous search space. To apply the optimal policy in real-world scenarios, we distill the augmentation process to speed up the preprocess. Extensive evaluations across diverse datasets and domains show our method significantly improves model adaptation with a domain-specific augmentation strategy. We will release our code to support further research.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on11
- Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao et al.ICCV 2023 · 13,211 citations
- Segment Everything Everywhere All at OnceXueyan Zou, Jianwei Yang, Hao Zhang, Feng Li et al.NeurIPS 2023 · 889 citations
- Depth Anything: Unleashing the Power of Large-Scale Unlabeled DataLihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu et al.CVPR 2024 · 847 citations
- Rethinking Coarse-to-Fine Approach in Single Image DeblurringSung-Jin Cho, Seo-Won Ji, Jun-Pyo Hong, Seung-Won Jung et al.ICCV 2021 · 799 citations
- Retinexformer: One-stage Retinex-based Transformer for Low-light Image EnhancementYuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang et al.ICCV 2023 · 615 citations
Related papers
- Prompting to Adapt Foundational Segmentation ModelsJie Hu, Jie Li, Yue Ma, Liujuan Cao et al.ACM MM 2024 · 1 citation
- Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot SegmentationJonas HerzogCVPR 2024
- PrAda: Few-Shot Visual Adaptation for Text-Prompted SegmentationGabriele Rosi, Fabio Cermelli, Carlo Masone, Barbara CaputoCVPR Findings 2026
- Cross-Domain Few-Shot Segmentation via Multi-view Progressive AdaptationJiahao Nie, Guanqiao Fu, Wenbin An, Yap-Peng Tan et al.CVPR 2026
- Primitive Generation and Semantic-Related Alignment for Universal Zero-Shot SegmentationShuting He, Henghui Ding, Wei JiangCVPR 2023
