WildSci: Advancing Scientific Reasoning from In-the-Wild Literature

Recent progress in large language model (LLM) reasoning has focused on domains like mathematics and coding, where abundant high-quality data and objective evaluation metrics are readily available. In contrast, progress in LLM reasoning models remains limited in scientific domains such as medicine and materials science due to limited dataset coverage and the inherent complexity of open-ended scientific questions. To address these challenges, we introduce WildSci, a new dataset of domain-specific science questions automatically synthesized from peer-reviewed literature, covering 9 scientific disciplines and 26 subdomains. By framing complex scientific reasoning tasks in a multiple-choice format, we enable scalable training with well-defined reward signals. We further apply reinforcement learning to finetune models on these data and analyze the resulting training dynamics, including domain-specific performance changes, response behaviors, and generalization trends. Experiments on a suite of scientific benchmarks demonstrate the effectiveness of our dataset and approach. We release WildSci to enable scalable and sustainable research in scientific reasoning. 1 Dataset Source Domains # Q Avg. Len. New? Camel-AI Science GPT-4 (self-generated) Phys, Bio, Chem 60K 30 ± 15 Yes Sci-Instruct Textbooks, problem sets, websites Phys, Chem, Math, Formal Proofs 254K 41 ± 33 No SCP-116K Educational materials Phys, Chem, Bio 116K 62 ± 74 No Natural Reasoning Pretraining corpora Multiple 2.8M 55 ± 21 Yes WildSci Peer-reviewed papers Multiple (research focused) 56K 82 ± 19 Yes textbooks or problem sets, scientific papers reflect the depth, rigor, and complexity of real-world research, making them well-suited for advancing models toward research-level reasoning skills. This approach offers several key advantages: (1) it grounds questions in real-world applications and expert-validated context; and (2) it enables the creation of new questions that are unlikely to appear in pretraining corpora, helping mitigate issues of data contamination. Another key challenge arises from the nature of science itself: many science questions are inherently open-ended and do not have a single verifiable answer. For example, explaining the observed decline in species richness in Figure 1 requires scientific judgment including interpreting evidence, reasoning about underlying mechanisms, and constructing plausible explanations. To address this, we adopt a more structured formulation by framing scientific reasoning tasks as multiple-choice questions (MCQs). MCQs are widely used in existing science benchmarks and offer a practical format for evaluation [Hendrycks et al., 2021a, Rein et al., 2023, Wang et al., 2024b, Team et al., 2025b]. This structure provides clear supervision signals, making it easier to define rewards systematically while preserving the richness of scientific reasoning. This simple setting offers a natural testbed for extending RL advances from mathematical domains to scientific reasoning tasks. In this work, we develop a generalizable approach for creating training data grounded in real-world scientific research, and to extend RLVR reasoning to scientific domains. Our contributions are summarized as follows. (1) We introduce a fully automated data synthesis pipeline that generates domain-specific questions from peer-reviewed scientific papers, followed by refinement and model voting to ensure data quality. (2) We construct WildSci, a dataset of 56K questions spanning 9 scientific disciplines and 26 subdomains, providing broad and diverse coverage for scientific reasoning. (3) We provide comprehensive analysis on how WildSci enables effective transfer of RLVR method to scientific domains. Models trained on WildSci show consistent improvements on multiple science benchmarks, including GPQA, SuperGPQA, and MMLU-Pro. With new papers continuously emerging in the community, WildSci provides a sustainable data synthesis approach to support ongoing exploration of scientific reasoning. We have open-sourced the code and data for WildSci to enable scalable and sustainable research in scientific reasoning. WildSci Data Creation An overview of our data creation pipeline is illustrated in Figure 1 . The entire process is fully automated using large language models, with multiple stages of filtering and refinement to ensure high-quality outputs. This automation enables our pipeline to generalize seamlessly to other scientific domains with accessible research literature. Peer-reviewed Papers We use publicly available, open-access articles from Nature Communications 2 as the data source [Li et al., 2024]. The journal categorizes its content into five major areas and 72 subdomains. We reorganize these into nine broader disciplines following the taxonomy of SuperGPQA [Team et al., 2025b]. To ensure balanced coverage, we randomly sample a subset of papers from each category and generate three questions per paper based on its content. While the