Interactive Segmentation as Gaussian Process Classification
Minghao Zhou, Hong Wang, Qian Zhao, Yuexiang Li, Yawen Huang, Deyu Meng, Yefeng Zheng
Abstract
Click-based interactive segmentation (IS) aims to extract the target objects under user interaction. For this task, most of the current deep learning (DL)-based methods mainly follow the general pipelines of semantic segmentation. Albeit achieving promising performance, they do not fully and explicitly utilize and propagate the click information, inevitably leading to unsatisfactory segmentation results, even at clicked points. Against this issue, in this paper, we propose to formulate the IS task as a Gaussian process (GP)-based pixel-wise binary classification model on each image. To solve this model, we utilize amortized variational inference to approximate the intractable GP posterior in a data-driven manner and then decouple the approximated GP posterior into double space forms for efficient sampling with linear complexity. Then, we correspondingly construct a GP classification framework, named GPCIS, which is integrated with the deep kernel learning mechanism for more flexibility. The main specificities of the proposed GPCIS lie in: 1) Under the explicit guidance of the derived GP posterior, the information contained in clicks can be finely propagated to the entire image and then boost the segmentation; 2) The accuracy of predictions at clicks has good theoretical support. These merits of GPCIS as well as its good generality and high efficiency are substantiated by comprehensive experiments on several benchmarks, as compared with representative methods both quantitatively and qualitatively. Codes will be released at https://github.com/zmhhmz/GPCIS CVPR2023.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
5 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Variance-Insensitive and Target-Preserving Mask Refinement for Interactive Image Segmentation cites FocalClick: Towards Practical Interactive Image Segmentation
- Variance-Insensitive and Target-Preserving Mask Refinement for Interactive Image Segmentation cites Conditional Diffusion for Interactive Segmentation
- Variance-Insensitive and Target-Preserving Mask Refinement for Interactive Image Segmentation cites AdaptIS: Adaptive Instance Selection Network
- Variance-Insensitive and Target-Preserving Mask Refinement for Interactive Image Segmentation cites SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
- FocalClick: Towards Practical Interactive Image Segmentation cites Conditional Diffusion for Interactive Segmentation
- FocalClick: Towards Practical Interactive Image Segmentation cites SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
- Multi-granularity Interaction Simulation for Unsupervised Interactive Segmentation cites FocalClick: Towards Practical Interactive Image Segmentation
- Multi-granularity Interaction Simulation for Unsupervised Interactive Segmentation cites Conditional Diffusion for Interactive Segmentation
- Repurposing Stable Diffusion Attention for Training-Free Unsupervised Interactive Segmentation cites FocalClick: Towards Practical Interactive Image Segmentation
- Repurposing Stable Diffusion Attention for Training-Free Unsupervised Interactive Segmentation cites Multi-granularity Interaction Simulation for Unsupervised Interactive Segmentation
- PaintSeg: Painting Pixels for Training-free Segmentation cites Multi-granularity Interaction Simulation for Unsupervised Interactive Segmentation
- Learning and Aligning Click-Aware Shape Prior for Interactive Amodal Instance Segmentation cites Variance-Insensitive and Target-Preserving Mask Refinement for Interactive Image Segmentation
- Learning and Aligning Click-Aware Shape Prior for Interactive Amodal Instance Segmentation cites FocalClick: Towards Practical Interactive Image Segmentation
Cited by top-tier papers5
- Multi-granularity Interaction Simulation for Unsupervised Interactive SegmentationKehan Li, Yian Zhao, Zhennan Wang, Zesen Cheng et al.ICCV 2023 · 11 citations
- PaintSeg: Painting Pixels for Training-free SegmentationXiang Li, Chung-Ching Lin, Yinpeng Chen, Zicheng Liu et al.NeurIPS 2023 · 8 citations
- Variance-Insensitive and Target-Preserving Mask Refinement for Interactive Image SegmentationChaowei Fang, Ziyin Zhou, Junye Chen, Hanjing Su et al.AAAI 2024 · 7 citations
- Repurposing Stable Diffusion Attention for Training-Free Unsupervised Interactive SegmentationMarkus Karmann, Onay UrfaliogluCVPR 2025
- Learning and Aligning Click-Aware Shape Prior for Interactive Amodal Instance SegmentationJunjie Chen, Junwei Lin, Ren Hong, Shengjie Liu et al.CVPR 2026
Builds on8
- SegFormer: Simple and Efficient Design for Semantic Segmentation with TransformersEnze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar et al.NeurIPS 2021 · 9,661 citations
- Efficiently sampling functions from Gaussian process posteriorsJames T. Wilson, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky et al.ICML 2020 · 186 citations
- AdaptIS: Adaptive Instance Selection NetworkKonstantin Sofiiuk, Olga Barinova, Anton KonushinICCV 2019 · 179 citations
- FocalClick: Towards Practical Interactive Image SegmentationXi Chen, Zhiyan Zhao, Yilei Zhang, Manni Duan et al.CVPR 2022 · 153 citations
- Conditional Diffusion for Interactive SegmentationXi Chen, Zhiyan Zhao, Feiwu Yu, Yilei Zhang et al.ICCV 2021 · 100 citations
Related papers
- Efficient Mask Correction for Click-Based Interactive Image SegmentationFei Du, Jianlong Yuan, Zhibin Wang, Fan WangCVPR 2023
- A Continual Learning Framework for Uncertainty-Aware Interactive Image SegmentationErvine Zheng, Qi Yu, Rui Li, Pengcheng Shi et al.AAAI 2021 · 27 citations
- Focused and Collaborative Feedback Integration for Interactive Image SegmentationQiaoqiao Wei, Hui Zhang, Jun-Hai YongCVPR 2023
- Order-aware Interactive SegmentationBin Wang, Anwesa Choudhuri, Meng Zheng, Zhongpai Gao et al.ICLR 2025
- Probabilistic Interactive 3D Segmentation with Hierarchical Neural ProcessesJie Liu, Pan Zhou, Zehao Xiao, Jiayi Shen et al.ICML 2025
