PSDPM: Prototype-based Secondary Discriminative Pixels Mining for Weakly Supervised Semantic Segmentation

Image-level Weakly Supervised Semantic Segmentation (WSSS) has received increasing attention due to its low an-notation cost. Class Activation Mapping (CAM) generated through classifier weights in WSSS inevitably ignores cer-tain useful cues, while the CAM generated through class prototypes can alleviate that. However, because of the dif-ferent goals of image classification and semantic segmentation, the class prototypes still focus on activating primary discriminative pixels learned from classification loss, leading to incomplete CAM. In this paper, we propose a plug-and-play Prototype-based Secondary Discriminative Pixels Mining (PSDPM) framework for enabling class prototypes to activate more secondary discriminative pixels, thus gen-erating a more complete CAM. Specifically, we introduce a Foreground Pixel Estimation Module (FPEM) for esti-mating potential foreground pixels based on the correlations between primary and secondary discriminative pix-els and the semantic segmentation results of baseline meth-ods. Then, we enable WSSS model to learn discriminative features from secondary discriminative pixels through a consistency loss calculated between FPEM result and class-prototype CAM. Experimental results show that our PSDPM improves various baseline methods significantly and achieves new state-of-the-art performances on WSSS benchmarks. Codes are available at https://github.com/xinqiaozhao/PSDPM.