Lune

ACL Findings2021

On the Cost-Effectiveness of Stacking of Neural and Non-Neural Methods for Text Classification: Scenarios and Performance Prediction

Christian Gomes, Marcos André Gonçalves, Leonardo Rocha, Sérgio D. Canuto

Abstract

Neural network algorithms such as those based on transformers and attention models have excelled on Automatic Text Classification (ATC) tasks. However, such enhanced performance comes at high computational costs. Ensembles of simpler classifiers (i.e., stacking) that exploit algorithmic and representational complementarities have also been shown to produce top-notch performance in ATC, enjoying high effectiveness and potentially lower computational costs. In this context, we present the first and largest comparative study to exploit the cost-effectiveness of stacking of ATC classifiers, consisting of transformers and non-neural algorithms. In particular, we are interested in answering research questions such as: Is it possible to obtain an effective ensemble with significantly less computational cost than the best learning model for a given dataset? Disregarding the computational cost, can an ensemble improve the effectiveness of the best learning model? Besides answering such questions, another main contribution of this paper is the proposal of a low-cost oracle-based method that can predict the best ensemble in each scenario (with and without computational cost limitations) using only a fraction of the available training data.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Related papers

  1. Revisiting Uncertainty-based Query Strategies for Active Learning with TransformersChristopher Schröder, Andreas Niekler, Martin PotthastACL Findings 2022 · 57 citations
  2. ACT: an Attentive Convolutional Transformer for Efficient Text ClassificationPengfei Li, Peixiang Zhong, Kezhi Mao, Dongzhe Wang et al.AAAI 2021 · 47 citations
  3. Fusing Label Embedding into BERT: An Efficient Improvement for Text ClassificationYijin Xiong, Yukun Feng, Hao Wu, Hidetaka Kamigaito et al.ACL Findings 2021 · 56 citations
  4. KwikBucks: Correlation Clustering with Cheap-Weak and Expensive-Strong SignalsSandeep Silwal, Sara Ahmadian, Andrew Nystrom, Andrew McCallum et al.ICLR 2023 · 3 citations
  5. Few-Shot Learning with Siamese Networks and Label TuningThomas Müller, Guillermo Pérez-Torró, Marc Franco-SalvadorACL 2022
Dusk over the sea between two cliffs drawn in fine vertical lines