Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Model

Natural language generation models reproduce and often amplify the biases present in their training data. Previous research explored using sequence-to-sequence rewriting models to transform biased model outputs (or original texts) into more gender-fair language by creating pseudo training data through linguistic rules. However, this approach is not practical for languages with more complex morphology than English. We hypothesise that creating training data in the reverse direction, i.e. starting from gender-fair text, is easier for morphologically complex languages and show that it matches the performance of state-of-the-art rewriting models for English. To eliminate the rule-based nature of data creation, we instead propose using machine translation models to create gender-biased text from real gender-fair text via round-trip translation. Our approach allows us to train a rewriting model for German without the need for elaborate handcrafted rules. The outputs of this model increased genderfairness as shown in a human evaluation study. 1 * Work done during an internship at Textshuttle. 1 We publicly release our data and code here: https:// github.com/textshuttle/exploiting-bias-to-debias Issue 1: Rewritings that affect other dependencies, e.g. when rewriting third-person subject pronouns in English, verbs in present tense need to be pluralised (e.g. "she knows" rewritten as "they know").