ACL Findings2023
Cross Encoding as Augmentation: Towards Effective Educational Text Classification
Hyun Seung Lee, Seungtaek Choi, Yunsung Lee, Hyeongdon Moon, Shinhyeok Oh, Myeongho Jeong, Hyojun Go, Christian Wallraven
Abstract
Text classification in education, usually called auto-tagging, is the automated process of assigning relevant tags to educational content, such as questions and textbooks. However, auto-tagging suffers from a data scarcity problem, which stems from two major challenges: 1) it possesses a large tag space and 2) it is multi-label. Though a retrieval approach is reportedly good at low-resource scenarios, there have been fewer efforts to directly address the data scarcity problem. To mitigate these issues, here we propose a novel retrieval approach CEAA that provides effective learning in educational text classification. Our main contributions are as follows: 1) we leverage transfer learning from question-answering datasets, and 2) we propose a simple but effective data augmentation method introducing cross-encoder style texts to a bi-encoder architecture for more efficient inference. An extensive set of experiments shows that our proposed method is effective in multi-label scenarios and low-resource tags compared to state-of-the-art models.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers1
Builds on4
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence ScoringSamuel Humeau, Kurt Shuster, Marie-Anne Lachaux, Jason WestonICLR 2020 · 316 citations
- Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis et al.EMNLP 2020 · 142 citations
- An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot LabelsIlias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas, Prodromos Malakasiotis et al.EMNLP 2020 · 2 citations
Related papers
- Text AutoAugment: Learning Compositional Augmentation Policy for Text ClassificationShuhuai Ren, Jinchao Zhang, Lei Li, Xu Sun et al.EMNLP 2021 · 22 citations
- Text Augmentation Using Dataset Reconstruction for Low-Resource ClassificationAdir Rahamim, Guy Uziel, Esther Goldbraich, Ateret Anaby-TavorACL Findings 2023 · 8 citations
- Unsupervised Energy-based Adversarial Domain Adaptation for Cross-domain Text ClassificationHan Zou, Jianfei Yang, Xiaojian WuACL Findings 2021 · 30 citations
- Style Transfer as Data Augmentation: A Case Study on Named Entity RecognitionShuguang Chen, Leonardo Neves, Thamar SolorioEMNLP 2022 · 9 citations
- Don't Miss the Labels: Label-semantic Augmented Meta-Learner for Few-Shot Text ClassificationQiaoyang Luo, Lingqiao Liu, Yuhao Lin, Wei ZhangACL Findings 2021 · 52 citations
