Lune

ACL Findings2026

ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decomposition

Yujie Liu, Zonglin Yang, Tong Xie, Jinjie Ni, Ben Gao, Yuqiang Li, Shixiang Tang, Wanli Ouyang, Erik Cambria, Dongzhan Zhou

Abstract

Large language models (LLMs) have shown potential in assisting scientific research, yet their ability to discover high-quality research hypotheses remains unexamined due to the lack of a dedicated benchmark. To address this gap, we introduce the first large-scale benchmark for evaluating LLMs on a sufficient set of scientific discovery sub-tasks-inspiration retrieval, hypothesis composition, and hypothesis ranking-where sufficient means that perfectly solving these sub-tasks perfectly solves the overall discovery task. We develop an automated LLM-based framework that extracts critical components-research questions, background surveys, inspirations, and hypotheses-from papers across 12 disciplines, with expert validation confirming its accuracy. To prevent data contamination, we focus exclusively on publications from 2024 onward, ensuring minimal overlap with LLM pretraining data; our automated framework further enables automatic extraction of even more recent papers as LLM pretraining cutoffs advance, supporting scalable and contamination-free automatic renewal of this discovery benchmark. Our evaluation shows that, across disciplines, LLMs excel at inspiration retrieval-an out-of-distribution task-suggesting their ability to surface novel knowledge associations.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

11 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents cites Deep Research Arena: The First Exam of LLMs' Research Abilities via Seminar-Grounded Tasks
  • MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search cites SciMON: Scientific Inspiration Machines Optimized for Novelty
  • MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures cites Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
  • From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery cites SciMON: Scientific Inspiration Machines Optimized for Novelty
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers11

  1. ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research AgentsManasi Sharma, Chen Bo Calvin Zhang, Chaithanya Bandi, Clinton Wang et al.ICLR 2026 · 83 citations
  2. Deep Research Arena: The First Exam of LLMs' Research Abilities via Seminar-Grounded TasksHaiyuan Wan, Chen Yang, Junchi Yu, Meiqi Tu et al.AAAI 2026 · 20 citations
  3. MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical SearchZonglin Yang, Wanhao Liu, Ben Gao, Yujie Liu et al.NeurIPS 2025 · 19 citations
  4. From Automation to Autonomy: A Survey on Large Language Models in Scientific DiscoveryTianshi Zheng, Zheye Deng, Hong Ting Tsang, Weiqi Wang et al.EMNLP 2025 · 5 citations
  5. FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific InsightsZhen Wang, Fan Bai, Zhongyan Luo, Jinyan Su et al.ICML 2026 · 2 citations

Builds on9

  1. Self-Refine: Iterative Refinement with Self-FeedbackAman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan et al.NeurIPS 2023 · 4,972 citations
  2. Chatbot Arena: An Open Platform for Evaluating LLMs by Human PreferenceWei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos et al.ICML 2024 · 1,212 citations
  3. MixEval: Deriving Wisdom of the Crowd from LLM Benchmark MixturesJinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng et al.NeurIPS 2024 · 88 citations
  4. DeepScientist: Advancing Frontier-Pushing Scientific Findings ProgressivelyYixuan Weng, Minjun Zhu, Qiujie Xie, Qiyao Sun et al.ICLR 2026 · 57 citations
  5. SciMON: Scientific Inspiration Machines Optimized for NoveltyQingyun Wang, Doug Downey, Heng Ji, Tom HopeACL 2024 · 22 citations

Related papers

  1. DiscoveryBench: Towards Data-Driven Discovery with Large Language ModelsBodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Bhavana Dalvi Mishra et al.ICLR 2025
  2. MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific HypothesesZonglin Yang, Wanhao Liu, Ben Gao, Tong Xie et al.ICLR 2025 · 2 citations
  3. LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language ModelsParshin Shojaee, Ngoc-Hieu Nguyen, Kazem Meidani, Amir Barati Farimani et al.ICML 2025
  4. ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific DiscoveryZiru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang et al.ICLR 2025 · 6 citations
  5. InsightEval: An Expert-Curated Benchmark for Assessing Insight Discovery in LLM-Driven Data AgentsZhenghao Zhu, Yuanfeng Song, Xing Chen, Chengzhong Liu et al.ACL Findings 2026
Dusk over the sea between two cliffs drawn in fine vertical lines