CreAItive Collaboration? Users' Misjudgment of AI-Creativity Affects Their Collaborative Performance

How does generative AI affect collaborative creative work and humans’ capability to carry it out? We tested 52 participant pairs in a standard creativity test, the Alternate Uses Test. The experimental AI group had access to ChatGPT-4, while the control group did not. The intervention did not lead to an improved performance overall. Further, the AI group elaborated their ideas significantly less. This effect carried over to the unaided post-test, pointing to longer-term effects of AI be(com)ing everyday technology, as how people perform a task with a tool shapes how they (learn to) perform the task without it. Analysis of the human-AI collaboration process revealed that participants were selective in using ChatGPT-4 output for the experimental task, misjudging and falsely assessing its output. This actually reduced their number of created ideas and underscores that users need to understand a (generative AI-based) tool’s capability for the specific task to support effective performance.