Preserving Label Correlation for Multi-label Text Classification by Prototypical Regularizations

Multi-label text classification (MLTC) aims to assign multiple relevant labels to a given sentence. An inherent challenge of MLTC is capturing label correlations compared with multi-class text classification. Existing MLTC models primarily focus on leveraging correlation information but often overlook the common issue of overfitting. Meanwhile, plug-and-play regularization methods struggle to preserve correlations effectively. In this paper, we distinguish two types of label correlations: explicit co-occurring correlation and implicit semantic correlations, and propose two regularization methods based on prototypical label embeddings for two correlation preservation, respectively. Specifically, we first generate the prototypical label embedding of multiple co-occurred labels as an intermediate. We then apply a prototypical label regularization on the distance between the sentence embedding and corresponding prototypical label embedding to alleviate the over-alignment issue caused by binary cross entropy loss and facilitate explicit correlation preservation. We finally extend the vanilla Mixup, which solely mixes multi-hot labels, on prototypical label embedding mixing to promote implicit correlation preservation. Empirical studies show the effectiveness of our regularization methods. CCS Concepts • Computing methodologies → Regularization.