CVPR Findings2026
Human-Intervention Segmentation via Federated Intent Embedding and Multi-Mask Recommendation
Yeongsu Kim, Seo-Yeon Choi, Kyungsu Lee
Abstract
Conventional image segmentation assumes a fixed notion of ground truth, whereas in many real-world scenarios, users repeatedly adjust model outputs to subjective or domainspecific preferences. Human-in-the-loop methods address this issue by incorporating user input during inference, but typically require intervention for each prediction and do not retain feedback for future use. To overcome this limitation, a new segmentation task, Human-Intervention Segmentation, is introduced that aims to generate segmentation masks that reflect user intent rather than relying solely on objective ground truth. A federated, training-free, and non-parametric framework is proposed that encodes user feedback as persistent intent embeddings. These embeddings are reused to personalize future predictions without additional intervention. The system produces multiple segmentation outputs in a single forward pass: (1) a baseline mask, (2) a personalized mask conditioned on the current intent, and (3) several recommendation masks based on similar intent vectors retrieved from other users. Embeddings are shared across clients in a privacy-preserving federated setting, and a central server retrieves semantically relevant references to guide recommendations. This architecture enables scalable, collaborative personalization without requiring parameter updates and is particularly suitable across diverse domains.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Segment Anything cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation cites SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
- DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation cites SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Builds on12
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao et al.ICCV 2023 · 13,211 citations
- SegFormer: Simple and Efficient Design for Semantic Segmentation with TransformersEnze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar et al.NeurIPS 2021 · 9,661 citations
- SegNeXt: Rethinking Convolutional Attention Design for Semantic SegmentationMeng-Hao Guo, Cheng-Ze Lu, Qibin Hou, Zhengning Liu et al.NeurIPS 2022 · 1,385 citations
- DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic SegmentationLukas Hoyer, Dengxin Dai, Luc Van GoolCVPR 2022 · 562 citations
Related papers
- A Continual Learning Framework for Uncertainty-Aware Interactive Image SegmentationErvine Zheng, Qi Yu, Rui Li, Pengcheng Shi et al.AAAI 2021 · 27 citations
- Federated Context-Aware Personalized RecommendationZhihao Wang, Xiaoying Liao, Wenke Huang, Bingqian Liu et al.AAAI 2026
- Federated Black-Box Adaptation for Semantic SegmentationJay N. Paranjape, Shameema Sikder, S. Swaroop Vedula, Vishal M. PatelNeurIPS 2024 · 4 citations
- Decentralised Learning from Independent Multi-Domain Labels for Person Re-IdentificationGuile Wu, Shaogang GongAAAI 2021 · 39 citations
- Personalize Segment Anything Model with One ShotRenrui Zhang, Zhengkai Jiang, Ziyu Guo, Shilin Yan et al.ICLR 2024 · 333 citations
