When Help Hurts: Verification Load and Fatigue with AI Coding Assistants
Guangrui Fan, Dandan Liu, Lihu Pan, Rui Zhang
Abstract
AI coding assistants help, but developers still spend effort verifying model output. We isolate interface effects by holding a single LLM fixed while N = 60 participants solve three Python tasks with Inline, Chat, or Structured prompting, plus a no‑AI control. AI reduced workload by − 18.2 TLX points and time by 22% (25.0 vs. 32.1 min) and improved correctness (OR = 1.71). Within AI, Inline is fastest and lowest‑load on simple work; Chat yields higher correctness beyond a per‑observation complexity threshold (z ≈ + 0.41) without a time cost; Structured benefits novices at mid complexity. We introduce a mode‑agnostic verification‑load index (failures, time‑to‑first‑compile, churn, pauses, switches) that partially mediates rising stress/fatigue across tasks. We translate these findings into design guidance: adaptive mode orchestration, transparency on demand, and verification aware packaging, and propose reporting verification load alongside outcomes to evaluate interfaces as models evolve.
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Related papers
- How Do Analysts Understand and Verify AI-Assisted Data Analyses?Ken Gu, Ruoxi Shang, Tim Althoff, Chenglong Wang et al.CHI 2024 · 36 citations
- Validating AI-Generated Code with Live ProgrammingKasra Ferdowsi, Ruanqianqian (Lisa) Huang, Michael B. James, Nadia Polikarpova et al.CHI 2024 · 27 citations
- From Code Generation to Conceptual Learning: Student Use of LLMs in a Web Programming CourseHajara-Yasmin Isa, Matthew Weston, Muhammad Rizky Wellyanto, Ishita Karna et al.CHI 2026 · 1 citation
- Ivie: Lightweight Anchored Explanations of Just-Generated CodeLitao Yan, Alyssa Hwang, Zhiyuan Wu, Andrew HeadCHI 2024 · 40 citations
- Grounded Copilot: How Programmers Interact with Code-Generating ModelsShraddha Barke, Michael B. James, Nadia PolikarpovaOOPSLA 2023 · 408 citations
