RARE: Retrieval-Augmented Reasoning Modeling

Domain-specific intelligence demands specialized knowledge and sophisticated reasoning for problem-solving, posing significant challenges for large language models (LLMs) that struggle with knowledge hallucination and inadequate reasoning capabilities. For the efficient discovery and modeling of reasoning patterns, we propose Retrieval-Augmented Reasoning Modeling (RARE), a novel paradigm to extract and model reasoning patterns from data, represented as salient and learnable tokens. Specifically, RARE externalizes domain knowledge to retrievable sources and internalizes reasoning patterns in LLMs. By injecting retrieved knowledge into training prompts with masked losses, RARE transforms learning objectives from rote memorization to contextualized reasoning, making implicit reasoning patterns salient. It enables models to bypass parameter-intensive memorization and prioritize the modeling of higher-order reasoning patterns. Extensive experiments demonstrate that lightweight RARE-trained models (e.g., Llama-3.1-8B and Qwen3-8B) could achieve state-of-the-art performance, even rivaling retrieval-augmented Gemini-3-Pro-Preview and GPT-5.1 with trillion parameters. RARE catalyzes a paradigm shift where maintainable external knowledge bases synergize with compact, reasoning-optimized models, collectively driving more scalable domain-specific intelligence.