Criticmate: Stagewise Human-AI Co-Critique in Single-Screen UI Evaluation
Jisu Ko, Jinyoung Choi, Cielo Morales, Dajung Kim, Minsam Ko
Abstract
AI tools are increasingly used for UI evaluation, yet most treat evaluation as a single-pass, black-box process that limits both effective model reasoning and human involvement. Grounded in Situation Awareness (SA) theory, we reframe single-screen heuristic evaluation of mobile UIs as stagewise human–AI co-critique, structuring evaluation into three editable stages: Perception (what is on the screen), Comprehension (what elements mean and do), and Projection (what problems and fixes follow). We instantiate this framing in Criticmate, an interactive system that exposes intermediate reasoning artifacts for intervention. Across offline benchmarks and a controlled user study, we show that stagewise co-critique yields more expert-like and better balanced critiques than single-pass approaches, while supporting higher trust and engagement without reducing perceived autonomy.
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Your agent calls
Lunesearch_papers
Free to start. No credit card required.
Terminal
Install the CLIlune papers get d3a6db53-27b7-451a-baa1-12e4eeb99f70Related papers
- CritiqueCrew: Orchestrating Multi-Perspective Conversational Design CritiqueXiaojiao Chen, Jiahuan Zhou, Yunfeng Shu, Ruihan Wang et al.CHI 2026 · 2 citations
- "It Became My Buddy, But I'm Not Afraid to Disagree": A Multi-Session Study of UX Evaluators Collaborating with Conversational AI AssistantsEmily Kuang, Ehsan Jahangirzadeh Soure, Luyao Shen, Nitesh Goyal et al.CHI 2026 · 3 citations
- What Makes Creators Engage with Online Critiques? Understanding the Role of Artifacts' Creation Stage, Characteristics of Community Comments, and their InteractionsQingyu Guo, Chao Zhang, Hanfang Lyu, Zhenhui Peng et al.CHI 2023 · 10 citations
- Sensemaking With/About AI: Unpacking Design Professionals' Data Sensemaking Styles in a High-Stakes Industrial ContextYi Luo, Dimitrios Gkouskos, Nancy L. RussoCHI 2026 · 1 citation
- When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI CollaborationQuan Shi, Carlos E. Jimenez, Shunyu Yao, Nick Haber et al.NeurIPS 2025 · 5 citations
