Lune

EMNLP Findings2023

Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification

Jiahuan Yan, Haojun Gao, Kai Zhang, Weize Liu, Danny Chen, Jian Wu, Jintai Chen

Abstract

Deep learning approaches exhibit promising performances on various text tasks. However, they are still struggling on medical text classification since samples are often extremely imbalanced and scarce. Different from existing mainstream approaches that focus on supplementary semantics with external medical information, this paper aims to rethink the data challenges in medical texts and present a novel framework-agnostic algorithm called Text2Tree that only utilizes internal label hierarchy in training deep learning models. We embed the ICD code tree structure of labels into cascade attention modules for learning hierarchy-aware label representations. Two new learning schemes, Similarity Surrogate Learning (SSL) and Dissimilarity Mixup Learning (DML), are devised to boost text classification by reusing and distinguishing samples of other labels following the label representation hierarchy, respectively. Experiments on authoritative public datasets and real-world medical records show that our approach stably achieves superior performances over classical and advanced imbalanced classification methods. Our code is available at https://github.com/jyansir/Text2Tree. RO. I J K (a) The computing process of cascade attention layers Level 0 1 J J10 J12 J16 J12 J12.0 J12.8

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Supervised Contrastive Learning cites A Simple Framework for Contrastive Learning of Visual Representations
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on23

  1. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  2. Supervised Contrastive LearningPrannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna et al.NeurIPS 2020 · 7,049 citations
  3. Do Transformers Really Perform Badly for Graph Representation?Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng et al.NeurIPS 2021 · 1,632 citations
  4. Dice Loss for Data-imbalanced NLP TasksXiaoya Li, Xiaofei Sun, Yuxian Meng, Junjun Liang et al.ACL 2020 · 575 citations
  5. MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text ClassificationJiaao Chen, Zichao Yang, Diyi YangACL 2020 · 340 citations

Related papers

  1. HiTIN: Hierarchy-aware Tree Isomorphism Network for Hierarchical Text ClassificationHe Zhu, Chong Zhang, Junjie Huang, Junran Wu et al.ACL 2023 · 17 citations
  2. Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text ClassificationZihan Wang, Peiyi Wang, Lianzhe Huang, Xin Sun et al.ACL 2022 · 157 citations
  3. Hierarchical Text Classification as Sub-hierarchy Sequence GenerationSanghun Im, Gibaeg Kim, Heung-Seon Oh, Seongung Jo et al.AAAI 2023 · 11 citations
  4. MATCH: Metadata-Aware Text Classification in A Large HierarchyYu Zhang, Zhihong Shen, Yuxiao Dong, Kuansan Wang et al.WWW 2021 · 39 citations
  5. Towards Realistic Semi-supervised Medical Image ClassificationWenxue Li, Lie Ju, Feilong Tang, Peng Xia et al.AAAI 2025 · 8 citations
Dusk over the sea between two cliffs drawn in fine vertical lines