SciCheck: Reasoning Distillation for Biomedical Claim Verification

False claims about medical information can have damaging consequences. Although LLMs have been used to verify such claims, their success depends on access to accurate content and robust reasoning capabilities. Prior research assumes access to gold evidence during inference or relies on expensive, compute-intensive scaling strategies. However, these approaches fall short when applied to real-world verification tasks, where relevant evidence is hard to obtain and efficiency is crucial. To address this, we introduce SciCheck, a novel solution that integrates web evidence retrieval with a process of reasoning distillation. Our approach fine-tunes a small language model using reasoning traces generated by an LLM. The distillation process involves a data preparation pipeline that avoids data leakage and filters reasoning traces to retain only those leading to correct answers. It also combines web and gold evidence during training to improve robustness, while evaluation is performed with web retrieval only. We performed an extensive experimental evaluation on different claim verification datasets. The results demonstrate that SciCheck outperforms competing approaches and proprietary LLMs such as Gemini 2.5 Flash in most scenarios with lower computational cost.