Mitigating Gender Bias via Fostering Exploratory Thinking in LLMs

Large Language Models (LLMs) often exhibit gender bias, resulting in unequal treatment of male and female subjects across different contexts. In particular, our recent work (Bajaj et al., 2024) highlights that LLMs make one-sided opposite moral judgments depending on the gender of the main character for morally ambiguous scenarios. Inspired by this finding, we propose a novel data generation framework to mitigate gender bias that fosters exploratory thinking in LLMs. Our approach prompts an LLM to generate morally ambiguous story pairs featuring protagonists of a different gender in otherwise structurally identical scenarios. For the story scenarios where the model actually exhibits inconsistent moral reasoning based on gender, we prompt the model to produce neutral exploratory judgments that integrate both moral and immoral perspectives. These exploratory judgments are used as supervision to fine-tune the model or optimize it via Direct Preference Optimization (DPO). Experimental results show that our method effectively reduces gender bias, while preserving or even enhancing general model capabilities. We release the code and generated data at: https://github.com/WeiKangda/ LLMs-Exploratory-Bias-Mitigation/ tree/main.