Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference

It has been shown that NLI models are usually biased with respect to the word-overlap between premise and hypothesis; they take this feature as a primary cue for predicting the entailment label. In this paper, we focus on an overlooked aspect of the overlap bias in NLI models: the reverse word-overlap bias. Our experimental results demonstrate that current NLI models are highly biased towards the non-entailment label on instances with low overlap, and the existing debiasing methods, which are reportedly successful on existing challenge datasets, are generally ineffective in addressing this category of bias. We investigate the reasons for the emergence of the overlap bias and the role of minority examples in its mitigation. For the former, we find that the word-overlap bias does not stem from pre-training, and for the latter, we observe that in contrast to the accepted assumption, eliminating minority examples does not affect the generalizability of debiasing methods with respect to the overlap bias. All the code and relevant data are available at: https: //github.com/sara-rajaee/reverse_bias Overlap Sample Label Full (1.0) P: A little kid in blue is sledding down a snowy hill. H: A little kid in blue sledding. Entailment P: The young lady is giving the old man a hug. H: The young man is giving the old man a hug. Non-Entailment 12 13 = 0.923 P: A woman in a blue shirt and green hat looks up at the camera. H: A woman wearing a blue shirt and green hat looks at the camera Entailment 11 12 = 0.917 P: Two men in wheelchairs are reaching in the air for a basketball. H: Two women in wheelchairs are reaching in the air for a basketball. Non-Entailment 1 14 = 0.071 P: Several young people sit at a table playing poker. H: Youthful Human beings are gathered around a flat surface to play a card game. Entailment