EMNLP Findings2023
Instances and Labels: Hierarchy-aware Joint Supervised Contrastive Learning for Hierarchical Multi-Label Text Classification
Simon Chi Lok U, Jie He, Víctor Gutiérrez-Basulto, Jeff Z. Pan
Abstract
Hierarchical multi-label text classification (HMTC) aims at utilizing a label hierarchy in multi-label classification. Recent approaches to HMTC deal with the problem of imposing an overconstrained premise on the output space by using contrastive learning on generated samples in a semi-supervised manner to bring text and label embeddings closer. However, the generation of samples tends to introduce noise as it ignores the correlation between similar samples in the same batch. One solution to this issue is supervised contrastive learning, but it remains an underexplored topic in HMTC due to its complex structured labels. To overcome this challenge, we propose HJCL, a Hierarchy-aware Joint Supervised Contrastive Learning method that bridges the gap between supervised contrastive learning and HMTC. Specifically, we employ both instance-wise and label-wise contrastive learning techniques and carefully construct batches to fulfill the contrastive learning objective. Extensive experiments on four multi-path HMTC datasets demonstrate that HJCL achieves promising results and the effectiveness of Contrastive Learning on HMTC. Code and data are available at https://github.com/simonucl/HJCL.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning cites A Simple Framework for Contrastive Learning of Visual Representations
- Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning cites Supervised Contrastive Learning
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere cites A Simple Framework for Contrastive Learning of Visual Representations
- Supervised Contrastive Learning cites Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
- Supervised Contrastive Learning cites A Simple Framework for Contrastive Learning of Visual Representations
Cited by top-tier papers3
- TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal SupervisionYunyi Zhang, Ruozhen Yang, Xueqiang Xu, Rui Li et al.WWW 2025 · 53 citations
- LGSA: Label Geometry Structuring and Aligning for Hierarchical Text ClassificationShuai Zhang, Weibo Xu, Jiahao Nie, Kecheng HuangACL 2026
- HYDRA: A Multi-Head Encoder-only Architecture for Hierarchical Text ClassificationFabian Karl, Ansgar ScherpEMNLP 2025
Builds on22
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- Supervised Contrastive LearningPrannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna et al.NeurIPS 2020 · 7,049 citations
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the HypersphereTongzhou Wang, Phillip IsolaICML 2020 · 2,360 citations
- Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningBeliz Gunel, Jingfei Du, Alexis Conneau, Veselin StoyanovICLR 2021 · 595 citations
Related papers
- Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text ClassificationZihan Wang, Peiyi Wang, Lianzhe Huang, Xin Sun et al.ACL 2022 · 157 citations
- Use All The Labels: A Hierarchical Multi-Label Contrastive Learning FrameworkShu Zhang, Ran Xu, Caiming Xiong, Chetan RamaiahCVPR 2022 · 71 citations
- MHCCL: Masked Hierarchical Cluster-Wise Contrastive Learning for Multivariate Time SeriesQianwen Meng, Hangwei Qian, Yong Liu, Lizhen Cui et al.AAAI 2023 · 54 citations
- Enhancing Hierarchical Text Classification through Knowledge Graph IntegrationYe Liu, Kai Zhang, Zhenya Huang, Kehang Wang et al.ACL Findings 2023 · 26 citations
- An Effective Deployment of Contrastive Learning in Multi-label Text ClassificationNankai Lin, Guanqiu Qin, Gang Wang, Dong Zhou et al.ACL Findings 2023 · 32 citations
