Toward Reliable Emotion Recognition: Alleviating Label Noise and Reducing Uncertain Prediction

Emotion recognition based on electroencephalogram (EEG) aims to recognize emotional states for improving user experience in Human-Computer Interaction, often using subjects' responses as labels. Physiological signals on widely used datasets (i.e. SEED, SEED-IV, and DEAP) are collected during subjects watching different types of movies as video stimulus. As a result, when subjects' emotional states change from one to another during a single video stimulus, two challenges are inevitable for reliable emotion recognition due to dynamic emotional fluctuations: (1) inaccurate annotation of EEG data; (2) feature confusion in classifier boundaries from similar emotional states (e.g., low-intensity happiness and neutral). However, previous studies have not given sufficient attention to the impact of the dynamic emotional fluctuations, leading to unreliable emotion recognition, especially in cross-subject emotion recognition scenarios. In this paper, we propose a Prototypes Collaborative Learning with Consistency Awareness (PCLCA) method to improve the reliability of cross-subject emotion recognition by introducing prototype learning. Specifically, a consistency awareness mechanism is designed to compute the consistency between labels and actual emotional states. Furthermore, a prototype collaborative strategy is adopted to adaptively estimate the uncertainty of model predictions by computing the similarity between features and prototypes. Extensive experiments on three benchmark datasets demonstrate that PCLCA effectively alleviates label noise and reduces uncertain predictions, outperforming existing baseline models.