Lune

ACL Findings2026

CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges

Zihan Wang, Lam Nguyen, Zhengyang Zhao, Mengyue Yang, Chengwei Qin, Yujiu Yang, Linyi Yang

Abstract

The saturation of high-quality pre-training data has shifted research focus toward evolutionary systems capable of continuously generating novel artifacts, leading to the success of AlphaEvolve. However, the progress of such systems is hindered by the lack of rigorous, quantitative evaluation. To tackle this challenge, we introduce CreativeBench, a benchmark for evaluating machine creativity in code generation, grounded in a classical cognitive framework. Comprising two subsets -CreativeBench-Combo and CreativeBench-Explore -the benchmark targets combinatorial and exploratory creativity through an automated pipeline utilizing reverse engineering and self-play. By leveraging executable code, CreativeBench objectively distinguishes creativity from hallucination via a unified metric defined as the product of quality and novelty. Our analysis of state-of-the-art models reveals distinct behaviors: (1) scaling significantly improves combinatorial creativity but yields diminishing returns for exploration; (2) larger models exhibit "convergence-by-scaling," becoming more correct but less divergent; and

(3) reasoning capabilities primarily benefit constrained exploration rather than combination. Finally, we propose EvoRePE, a plug-and-play inference-time steering strategy that internalizes evolutionary search patterns to consistently enhance machine creativity. We release our data and code at: CreativeBench Homepage.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • DiffoR: A Unified Continuous Generative Framework for Universal Ordinal Regression cites Language Models are Few-Shot Learners
  • Inference-Time Intervention: Eliciting Truthful Answers from a Language Model cites Language Models are Few-Shot Learners
  • Language Modeling Is Compression cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Learning from Contrasts: Synthesizing Reasoning Paths from Diverse Search TrajectoriesPeiyang Liu, Zhirui Chen, Xi Wang, Di Liang et al.ACL 2026 · 4 citations
  2. DiffoR: A Unified Continuous Generative Framework for Universal Ordinal RegressionHongxu Ma, Lin Wang, Chenghou Jin, Han Zhou et al.KDD 2026 · 1 citation

Builds on13

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. SWE-bench: Can Language Models Resolve Real-world Github Issues?Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao et al.ICLR 2024 · 2,082 citations
  3. Inference-Time Intervention: Eliciting Truthful Answers from a Language ModelKenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister et al.NeurIPS 2023 · 1,549 citations
  4. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement LearningLakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems et al.ICLR 2026 · 466 citations
  5. Language Modeling Is CompressionGrégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt et al.ICLR 2024 · 243 citations

Related papers

  1. EscapeBench: Towards Advancing Creative Intelligence of Language Model AgentsCheng Qian, Peixuan Han, Qinyu Luo, Bingxiang He et al.ACL 2025
  2. Creation-Mmbench: Assessing Context-Aware Creative Intelligence in MllmsXinyu Fang, Zhijian Chen, Kai Lan, Lixin Ma et al.ICCV 2025 · 23 citations
  3. Automated Creativity Evaluation of Language Models Across Open-Ended TasksTan Min Sen, Zachary Choy Kit Chun, Syed Ali Redha Alsagoff, Nadya Yuki Wangsajaya et al.ACL 2026
  4. CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative FeedbackQiushi Sun, Jingyang Gong, Lei Li, Qipeng Guo et al.ACL 2026 · 4 citations
  5. InsightEval: An Expert-Curated Benchmark for Assessing Insight Discovery in LLM-Driven Data AgentsZhenghao Zhu, Yuanfeng Song, Xing Chen, Chengzhong Liu et al.ACL Findings 2026
Dusk over the sea between two cliffs drawn in fine vertical lines