Algorithm-in-the-Loop Decision Making

Algorithms have entered the center of many decision-making processes: rather than being made solely by people, many important decisions are now made through an "algorithm-in-the-loop" process where algorithmic models inform people. Yet insufficient research has considered how the interactions between people and algorithms actually influence human decisions. In this talk, I'll discuss results from a set of controlled human-subject experiments on algorithm-in-the-loop human decision making. In our experiments in two contexts (pretrial release and financial lending), when presented with algorithmic risk assessments, our participants exhibited additional bias in their decisions, did not calibrate their reliance on the algorithm based on the algorithm's performance and showed a change in their decision-making process. These results highlight the urgent need to expand our analyses of algorithmic decision making aids beyond evaluating the algorithms themselves to investigating the full socio-technical contexts in which people and algorithms interact.