Out-of-Distributed Semantic Pruning for Robust Semi-Supervised Learning
Yu Wang, Pengchong Qiao, Chang Liu, Guoli Song, Xiawu Zheng, Jie Chen
Abstract
Recent advances in robust semi-supervised learning (SSL) typically filter out-of-distribution (OOD) information at the sample level. We argue that an overlooked problem of robust SSL is its corrupted information on semantic level, practically limiting the development of the field. In this paper, we take an initial step to explore and propose a unified framework termed OOD Semantic Pruning (OSP), which aims at pruning OOD semantics out from in-distribution (ID) features. Specifically, (i) we propose an aliasing OOD matching module to pair each ID sample with an OOD sample with semantic overlap. (ii) We design a soft orthogonality regularization, which first transforms each ID feature by suppressing its semantic component that is collinear with paired OOD sample. It then forces the predictions before and after soft orthogonality decomposition to be consistent. Being practically simple, our method shows a strong performance in OOD detection and ID classification on challenging benchmarks. In particular, OSP surpasses the previous state-of-the-art by 13.7% on accuracy for ID classification and 5.9% on AUROC for OOD detection on TinyIm-ageNet dataset. The source codes are publicly available at https://github.com/rain305f/OSP .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
8 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- MOS: Modeling Object-Scene Associations in Generalized Category Discovery cites Discover and Align Taxonomic Context Priors for Open-world Semi-Supervised Learning
- MOS: Modeling Object-Scene Associations in Generalized Category Discovery cites FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Unknown-Aware Graph Regularization for Robust Semi-supervised Learning from Uncurated Data cites A Simple Framework for Contrastive Learning of Visual Representations
- Unknown-Aware Graph Regularization for Robust Semi-supervised Learning from Uncurated Data cites Unsupervised Data Augmentation for Consistency Training
- Unknown-Aware Graph Regularization for Robust Semi-supervised Learning from Uncurated Data cites FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Binary Decomposition: A Problem Transformation Perspective for Open-Set Semi-Supervised Learning cites Unsupervised Data Augmentation for Consistency Training
- Binary Decomposition: A Problem Transformation Perspective for Open-Set Semi-Supervised Learning cites FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Big Self-Supervised Models are Strong Semi-Supervised Learners cites A Simple Framework for Contrastive Learning of Visual Representations
- Big Self-Supervised Models are Strong Semi-Supervised Learners cites FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Learning Transferable Visual Models From Natural Language Supervision cites A Simple Framework for Contrastive Learning of Visual Representations
- Learning Transferable Visual Models From Natural Language Supervision cites Big Self-Supervised Models are Strong Semi-Supervised Learners
- Discover and Align Taxonomic Context Priors for Open-world Semi-Supervised Learning cites Unsupervised Data Augmentation for Consistency Training
- Discover and Align Taxonomic Context Priors for Open-world Semi-Supervised Learning cites Big Self-Supervised Models are Strong Semi-Supervised Learners
- Discover and Align Taxonomic Context Priors for Open-world Semi-Supervised Learning cites Learning Transferable Visual Models From Natural Language Supervision
- Discover and Align Taxonomic Context Priors for Open-world Semi-Supervised Learning cites FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence cites Unsupervised Data Augmentation for Consistency Training
Cited by top-tier papers8
- Discover and Align Taxonomic Context Priors for Open-world Semi-Supervised LearningYu Wang, Zhun Zhong, Pengchong Qiao, Xuxin Cheng et al.NeurIPS 2023 · 25 citations
- Unknown-Aware Graph Regularization for Robust Semi-supervised Learning from Uncurated DataHeejo Kong, Suneung Kim, Ho-Joong Kim, Seong-Whan LeeAAAI 2024 · 7 citations
- Binary Decomposition: A Problem Transformation Perspective for Open-Set Semi-Supervised LearningJun-Yi Hang, Min-Ling ZhangICML 2024 · 4 citations
- Open-Set Heterogeneous Domain Adaptation: Theoretical Analysis and AlgorithmThai-Hoang Pham, Yuanlong Wang, Changchang Yin, Xueru Zhang et al.AAAI 2025 · 1 citation
- MOS: Modeling Object-Scene Associations in Generalized Category DiscoveryZhengyuan Peng, Jinpeng Ma, Zhimin Sun, Ran Yi et al.CVPR 2025
Builds on17
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and ConfidenceKihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang et al.NeurIPS 2020 · 5,129 citations
- Unsupervised Data Augmentation for Consistency TrainingQizhe Xie, Zihang Dai, Eduard H. Hovy, Thang Luong et al.NeurIPS 2020 · 2,774 citations
- Big Self-Supervised Models are Strong Semi-Supervised LearnersTing Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi et al.NeurIPS 2020 · 2,611 citations
Related papers
- Let the Void Be Void: Robust Open-Set Semi-Supervised Learning via Selective Non-AlignmentYou Rim Choi, Subeom Park, Seojun Heo, Eunchung Noh et al.AAAI 2026
- OpenMatch: Open-Set Semi-supervised Learning with Open-set Consistency RegularizationKuniaki Saito, Donghyun Kim, Kate SaenkoNeurIPS 2021 · 80 citations
- STEP: Out-of-Distribution Detection in the Presence of Limited In-Distribution Labeled DataZhi Zhou, Lan-Zhe Guo, Zhanzhan Cheng, Yufeng Li et al.NeurIPS 2021 · 41 citations
- Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised LearningJunkai Huang, Chaowei Fang, Weikai Chen, Zhenhua Chai et al.ICCV 2021 · 74 citations
- IOMatch: Simplifying Open-Set Semi-Supervised Learning with Joint Inliers and Outliers UtilizationZekun Li, Lei Qi, Yinghuan Shi, Yang GaoICCV 2023 · 47 citations
