Understanding User Sensemaking in Machine Learning Fairness Assessment Systems

A variety of systems have been proposed to assist users in detecting machine learning (ML) fairness issues. These systems approach bias reduction from a number of perspectives, including recommender systems, exploratory tools, and dashboards. In this paper, we seek to inform the design of these systems by examining how individuals make sense of fairness issues as they use different debiasing affordances. In particular, we consider the tension between de-biasing recommendations which are quick but may lack nuance and "what-if" style exploration which is time consuming but may lead to deeper understanding and transferable insights. Using logs, think-aloud data, and semi-structured interviews we find that exploratory systems promote a rich pattern of hypothesis generation and testing, while recommendations deliver quick answers which satisfy participants at the cost of reduced information exposure. We highlight design requirements and trade-offs in the design of ML fairness systems to promote accurate and explainable assessments. CCS CONCEPTS • Human-centered computing → User models; • Computing methodologies → Machine learning algorithms.