Counterfactual Inference for Text Classification Debiasing

Today's text classifiers inevitably suffer from unintended dataset biases, especially the document-level label bias and word-level keyword bias, which may hurt models' generalization. Many previous studies employed datalevel manipulations or model-level balancing mechanisms to recover unbiased distributions and thus prevent models from capturing the two types of biases. Unfortunately, they either suffer from the extra cost of data collection/selection/annotation or need an elaborate design of balancing strategies. Different from traditional factual inference in which debiasing occurs before or during training, counterfactual inference mitigates the influence brought by unintended confounders after training, which can make unbiased decisions with biased observations. Inspired by this, we propose a model-agnostic text classification debiasing framework -CORSAIR, which can effectively avoid employing data manipulations or designing balancing mechanisms. Concretely, CORSAIR first trains a base model on a training set directly, allowing the dataset biases "poison" the trained model. In inference, given a factual input document, COR-SAIR imagines its two counterfactual counterparts to distill and mitigate the two biases captured by the poisonous model. Extensive experiments demonstrate CORSAIR's effectiveness, generalizability and fairness. 1 * This work was partly done during Chen Qian's internship at Alibaba DAMO academy. Fuli Feng and Lijie Wen are the co-corresponding authors. 1 The code is available at https://github.com/ qianc62/Corsair .