Lune

CHI2022Top-tier venue

Toward User-Driven Algorithm Auditing: Investigating users' strategies for uncovering harmful algorithmic behavior

Alicia DeVos, Aditi Dhabalia, Hong Shen, Kenneth Holstein, Motahhare Eslami

Abstract

Recent work in HCI suggests that users can be powerful in surfacing harmful algorithmic behaviors that formal auditing approaches fail to detect. However, it is not well understood how users are often able to be so effective, nor how we might support more effective user-driven auditing. To investigate, we conducted a series of think-aloud interviews, diary studies, and workshops, exploring how users find and make sense of harmful behaviors in algorithmic systems, both individually and collectively. Based on our findings, we present a process model capturing the dynamics of and influences on users’ search and sensemaking behaviors. We find that 1) users’ search strategies and interpretations are heavily guided by their personal experiences with and exposures to societal bias; and 2) collective sensemaking amongst multiple users is invaluable in user-driven algorithm audits. We offer directions for the design of future methods and tools that can better support user-driven auditing.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

32 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations cites The Landscape and Gaps in Open Source Fairness Toolkits
  • A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations cites Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors
  • A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations cites End-User Audits: A System Empowering Communities to Lead Large-Scale Investigations of Harmful Algorithmic Behavior
  • A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations cites Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges
  • A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations cites Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI
  • Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors cites The Landscape and Gaps in Open Source Fairness Toolkits
  • Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors cites Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI
  • Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors cites Algorithmic Folk Theories and Identity: How TikTok Users Co-Produce Knowledge of Identity and Engage in Algorithmic Resistance
  • End-User Audits: A System Empowering Communities to Lead Large-Scale Investigations of Harmful Algorithmic Behavior cites Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors
  • Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling cites A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
  • Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling cites The Landscape and Gaps in Open Source Fairness Toolkits
  • Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling cites Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors
  • Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling cites Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI
  • Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges cites The Landscape and Gaps in Open Source Fairness Toolkits
  • Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges cites Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors
  • Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges cites Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI
  • Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges cites Algorithmic Folk Theories and Identity: How TikTok Users Co-Produce Knowledge of Identity and Engage in Algorithmic Resistance
  • Cruising Queer HCI on the DL: A Literature Review of LGBTQ+ People in HCI cites Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors
  • Cruising Queer HCI on the DL: A Literature Review of LGBTQ+ People in HCI cites Algorithmic Folk Theories and Identity: How TikTok Users Co-Produce Knowledge of Identity and Engage in Algorithmic Resistance
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers32

  1. Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI ChallengesQiaosi Wang, Michael Madaio, Shaun K. Kane, Shivani Kapania et al.CHI 2023 · 90 citations
  2. End-User Audits: A System Empowering Communities to Lead Large-Scale Investigations of Harmful Algorithmic BehaviorMichelle S. Lam, Mitchell L. Gordon, Danaë Metaxa, Jeffrey T. Hancock et al.CSCW 2022 · 77 citations
  3. Cruising Queer HCI on the DL: A Literature Review of LGBTQ+ People in HCIJordan Taylor, Ellen Simpson, Anh-Ton Tran, Jed R. Brubaker et al.CHI 2024 · 55 citations
  4. Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit ToolingVictor Ojewale, Ryan Steed, Briana Vecchione, Abeba Birhane et al.CHI 2025 · 46 citations
  5. A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness EvaluationsGlen Berman, Nitesh Goyal, Michael MadaioCHI 2024 · 40 citations

Builds on10

  1. Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AIMichael A. Madaio, Luke Stark, Jennifer Wortman Vaughan, Hanna M. WallachCHI 2020 · 428 citations
  2. Algorithmic Folk Theories and Identity: How TikTok Users Co-Produce Knowledge of Identity and Engage in Algorithmic ResistanceNadia Karizat, Daniel Delmonaco, Motahhare Eslami, Nazanin AndalibiCSCW 2021 · 338 citations
  3. Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic BehaviorsHong Shen, Alicia DeVos, Motahhare Eslami, Kenneth HolsteinCSCW 2021 · 156 citations
  4. Adaptive Folk Theorization as a Path to Algorithmic Literacy on Changing PlatformsMichael Ann DeVitoCSCW 2021 · 143 citations
  5. The Landscape and Gaps in Open Source Fairness ToolkitsMichelle Seng Ah Lee, Jatinder SinghCHI 2021 · 117 citations

Related papers

  1. Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work together to Surface Algorithmic Harms?Rena Li, Sara Kingsley, Chelsea Fan, Proteeti Sinha et al.CHI 2023 · 28 citations
  2. Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry PracticeWesley Hanwen Deng, Bill Boyuan Guo, Alicia DeVrio, Hong Shen et al.CHI 2023 · 73 citations
  3. WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AIWesley Hanwen Deng, Claire Wang, Howard Ziyu Han, Jason I. Hong et al.CSCW 2025 · 12 citations
  4. Sensemaking in User-Driven Algorithm Auditing: A Case Study on Gender Bias in an Image Captioning ModelBehnoosh Mohammadzadeh, Jules Françoise, Michèle Gouiffès, Baptiste CaramiauxCHI 2026 · 1 citation
  5. Learning AI Auditing: A Case Study of Teenagers Auditing a Generative AI ModelLuis Morales-Navarro, Michelle A. Gan, Evelyn Yu, Lauren Vogelstein et al.CSCW 2025 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines