MuseScorer: Idea Originality Scoring At Scale

An objective, face-valid method for scoring idea originality is to measure each idea's statistical infrequency within a population-an approach long used in creativity research. Yet, computing these frequencies requires manually bucketing idea rephrasings, a process that is subjective, labor-intensive, error-prone, and brittle at scale. We introduce MUSESCORER, a fully automated, psychometrically validated system for frequency-based originality scoring. MUSESCORER integrates a Large Language Model (LLM) with externally orchestrated retrieval: given a new idea, it retrieves semantically similar prior idea-buckets and zero-shot prompts the LLM to judge whether the idea fits an existing bucket or forms a new one. These buckets enable frequencybased originality scoring without human annotation. Across five datasets (N participants =1143, n ideas =16,294), MUSESCORER matches human annotators in idea clustering structure (AMI = 0.59) and participant-level scoring (r = 0.89), while demonstrating strong convergent and external validity. The system enables scalable, intent-sensitive, and human-aligned originality assessment for creativity research. 'infrequency' can be reliably operationalized. Second, we release an automated, interpretable scoring pipeline deployable across diverse open-ended ideation tasks, enabling creativity research at scale 1 . More broadly, MUSESCORER demonstrates how advanced NLP methods can address long-standing annotation challenges, providing validated tools that adjacent disciplines can adopt with confidence.