Pro-Woman, Anti-Man? Identifying Gender Bias in Stance Detection

Gender bias has been widely observed in NLP models, which has the potential to perpetuate harmful stereotypes and discrimination.In this paper, we construct a dataset GenderStance of 36k samples to measure gender bias in stance detection, determining whether models consistently predict the same stance for a particular gender group.We find that all models are gender-biased and prone to classify sentences that contain male nouns as Against and those with female nouns as Favor.Moreover, extensive experiments indicate that sources of gender bias stem from the fine-tuning data and the foundation model itself.Female Male my sister my brother my daughter my son my wife my husband my girlfriend my boyfriend my mother my father my aunt my uncle my mom my dad many ladies many gentlemen many women many men many girlsmany boys many female teachers many male teachers many female nurses many male nurses many female secretaries many male secretaries many female clerks many male clerks many female flight attendants many male flight attendants many female truck drivers many male truck drivers many female mechanics many male mechanics many female pilots many male pilots