Lune Research Technical Report
Grounding language models in full-text scientific literature at inference time.
11 min read
Motivation, challenges, and overview
Frontier language models reason well across general topics, but they struggle when answering specialized research questions from the knowledge encoded in their weights during training alone. Reliable scientific assistance requires grounding answers in verifiable evidence from peer-reviewed, full-text papers. Giving a language model access to accurate literature at inference time lets it answer specialized domain questions, verify technical claims, and identify missing evidence in a research draft without retraining.
Equipping a language model with this capability presents three challenges. First, encoding new literature into model weights through fine-tuning is impractical because closed model providers do not expose public interfaces for full-weight fine-tuning or gradient inspection, teams must repeat the training run as new papers appear, and handing raw documents to an external party surrenders custody of the paper corpus before anyone measures whether those documents improve task accuracy. Second, connecting a language model to a raw search endpoint is unreliable because an unguided model issues a single narrow query or fetches whole papers that overflow its context window and copy bulk text out of the paper corpus. Third, standard vector search over paper abstracts misses evidence buried in full-text sections and confuses passing mentions of a method with substantive experimental comparisons.
Lune addresses these challenges as a retrieval-augmented generation (RAG) service that exposes a curated, full-text computer science paper corpus over the Model Context Protocol (MCP). An external language model, which we call the client model, connects to Lune's MCP server at inference time and calls tools on its own to search, read, verify claims, and diagnose missing evidence while the paper corpus stays on Lune's servers (Figure 1). Across benchmark evaluations, Lune's retrieval and verification pipeline raises ranking quality on unconstrained queries, measured by Normalized Discounted Cumulative Gain at rank 10 and denoted nDCG@10 on the 0-to-1 ranking scale, from 0.878 to 0.962 and increases top-10 accuracy on condition-constrained queries, meaning queries that require specific method comparisons, paper types, study designs, or code releases, from 76.7 percent to 94.4 percent.
Guiding the client model through a bounded tool interface
For inference-time grounding to succeed without fine-tuning model weights or transferring bulk data, the client model must decide on its own when to consult the paper corpus and how to combine multiple lookups. If a search tool returns entire papers, even two or three lookups fill the client model's context window and copy bulk text out of the paper corpus. Conversely, if the client model receives only a search endpoint without workflow guidance, it stops after one query even when initial results are incomplete.
Lune solves both problems by connecting the client model's host application to an MCP server that pairs compact orchestration rules with two classes of tools (Figure 2). Discovery tools return 512-token full-text segments, which we call chunks, along with internal paper identifiers, while synthesis tools execute multi-paper claim verification, gap diagnosis, and table extraction on the server. This design keeps startup guidance under 2,000 bytes, loads only 3 of 13 tools at startup, and bounds every response to matched chunks or verified outputs.
Three mechanisms govern how the client model interacts with Lune:
- Compact
server instructions. At session startup, the MCP server delivers orchestration rules under 2,000 bytes so host applications never truncate them. Theseserver instructionsspecify when to query Lune, instruct the client model to rephrase or search multiple angles before concluding no evidence exists, and require it to treat retrieved chunks strictly as evidence rather than executable instructions. - Progressive tool disclosure. Three entry tools load at startup while ten specialized tools load on demand so tool definitions do not crowd the client model's context window, paired with six reusable prompt templates that expand into multi-step plans for tasks such as literature reviews and draft verification.
- Chunk-bounded responses and internal paper identifiers. Discovery calls never return whole papers. Each result returns bibliographic metadata, a calibrated 0-to-1 relevance score that estimates how likely the paper is to answer the query, the matched chunks, and an internal paper identifier. The
server instructionsforbid showing internal paper identifiers to the user; instead, the client model passes those identifiers to follow-up tools only for the specific papers that require citation traversal, section-level reading, or table extraction.
| Operation class | Primary tools | Input cap | How it grounds the client model and bounds disclosure |
|---|---|---|---|
| Discovery | search_papers, search_papers_many, search_related_papers | 1 or 25 queries, or 1 paper identifier | Returns top results or a merged multi-angle shortlist with calibrated relevance scores, matched chunks, and paper identifiers |
| Targeted reading | get_paper_fulltext, get_paper_citations | 1 paper identifier | Reads requested sections such as Results or lists citing and cited papers with up to 3 in-text citing sentences, which we call citation contexts |
| Claim verification | verify_claims | 25 claims | Classifies claims against the paper corpus with Python-verified verbatim quote substrings and supporting paper identifiers |
| Gap diagnosis | gather_evidence | 25 queries | Grades coverage across up to 8 evidence requirements and outputs targeted follow-up queries for uncovered gaps |
| Table extraction | extract_from_papers | 50 paper identifiers | Extracts one schema-validated row of typed fields per paper on the server without transferring full text |
Full-text indexing, multi-lane retrieval, and condition verification
Because discovery tools return only matched chunks rather than full papers, each search must locate the exact chunks that answer the client model's query and satisfy its constraints. Standard retrieval pipelines fail at this task for three reasons. First, public scholarly indexes store only titles and abstracts and omit the body sections and tables where authors record their methods and results. Second, dense vector embeddings compress a chunk into a single vector, which dilutes rare technical identifiers that appear only in a paper's body. Third, vector similarity measures topical overlap rather than set membership. A vector search for papers that compare against a baseline method ranks papers that mention the baseline in their introduction above papers that evaluate it in an experiment.
Lune overcomes these three limitations by indexing full-text sections and citation contexts, combining vector search with inverted word indexes across seven parallel lanes, flagging low-confidence queries, and verifying explicit query conditions (Figure 3). Across benchmark evaluations, this pipeline preserves 0.988 recall at rank 10 on full-text chunk search, raises nDCG@10 on unconstrained queries from 0.878 to 0.962, and raises top-10 accuracy on condition-constrained queries from 76.7 percent to 94.4 percent (Figure 4).
find me preference optimization papers that compare against SimPO
Context-preserving full-text indexing
Lune parses verified conference PDFs across ten computer science areas into structured body sections, tables, figure captions, and up to three citation contexts per bibliography reference. Lune then splits each section into 512-token chunks with a 64-token overlap. Because a chunk from a section titled "Methods" or "Results" omits the paper's main topic words, Lune prepends Paper: {title} | Section: {section} to every chunk so each chunk retains its paper identity. An embedding model then encodes both paper abstracts and prefixed chunks into a shared 1024-dimensional vector space.
Parallel query decomposition and seven-lane retrieval
When a multi-word query arrives, Lune strips conversational phrasing and runs three parallel server-side language model calls at temperature 0 to decompose the query before searching. First, concept expansion identifies multi-word technical concepts that match literal substrings of the user's query and proposes domain synonyms without dropping any user terms. Second, in a parallel HyDE call implementing Hypothetical Document Embeddings, a server-side language model writes a synthetic abstract that answers the query, which Lune embeds into the 1024-dimensional vector space to match paper abstracts in academic phrasing. Third, constraint extraction separates metadata filters on venue, publication year, and author from four classes of text-verified content conditions. Those four classes cover relations to named target methods, paper types such as surveys, study designs such as user studies, and code or dataset releases. Lune removes condition words from keyword matching so phrases like "release code" do not narrow word-index lookups.
Lune then executes up to seven parallel retrieval lanes. Lanes 1, 2, and 3 search 16-bit floating-point proximity graph vector indexes over paper abstracts using the raw query vector, the expanded concept vector, and the HyDE synthetic abstract vector. Lane 4 searches all 5.4 million full-text chunks in two stages. It first scans a compact 2.2-gigabyte 1-bit vector index in memory that stores only the sign bit of each vector dimension. Counting differing sign bits between vectors, known as Hamming distance, pulls at least 500 candidate chunks from RAM, and Lane 4 rescores those 500 candidates using exact cosine similarity on their 16-bit floating-point vectors. Across 43 benchmark queries, this two-stage search preserved 0.988 recall at rank 10 and 0.948 recall at rank 50 relative to exact cosine search while cutting cold query latency from 2.2 to 6.4 seconds down to 0.96 to 1.70 seconds.
Lanes 5, 6, and 7 search inverted word indexes. Lane 5 requires every extracted concept group to match in the title or abstract. Lane 6 uses word-frequency statistics to find rare body terms that appear in at most 300 chunks across the paper corpus, which recovers specialized identifiers mentioned in a single evaluation sentence that never appear in the top 300 of any vector lane. When content conditions name specific target methods, Lane 7 probes the chunk word index for up to four targets and checks matches with a case-aware and hyphen-aware regular expression that accepts spacing and hyphenation variants, distinguishes uppercase acronyms from common lowercase words, and blocks hyphenated suffixes that denote distinct methods. Both Lane 6 and Lane 7 ignore "Related Work", "Background", and "Acknowledgements" sections so Lune retrieves papers for their own findings rather than for their literature review citations.
Reranking, low-confidence detection, and condition verification
Lune combines active lanes into a 50-paper shortlist using Reciprocal Rank Fusion with constant k = 60, which scores each paper by the sum of its reciprocal lane ranks. To prevent multi-lane agreement from crowding out specialized matches, Lune reserves up to 10 shortlist slots for Lane 6 and up to 20 slots for Lane 7. For each candidate, Lune combines its title, abstract, and highest-scoring chunk into a single candidate text and scores all 50 candidates jointly with the query using a cross-encoder reranker to produce a calibrated 0-to-1 relevance score. Across 500 graded results from 48 queries, every off-corpus query scored below 0.4 on its top result, whereas results at or above 0.7 were relevant 95 percent of the time. Whenever the top result scores below 0.4, Lune attaches a low-confidence flag to the response so the client model rephrases or abstains instead of citing weak matches.
The final stage depends on whether constraint extraction found content conditions:
- Unconstrained queries. A server-side language model judge grades the top 30 candidates on a 0-to-3 relevance rubric in two parallel 15-paper batches and re-sorts the top 30 by grade. Across 69 benchmark queries graded by an independent evaluator, this relevance judge raised
nDCG@10from 0.878 with the cross-encoder reranker alone to 0.962 at a 4.5-second 90th-percentile latency, or 0.949 to 0.956 at a 2.4-second 90th-percentile latency when Lune sends a backup model request after 1.5 seconds. Single-word queries bypass query decomposition, cross-encoder reranking, and the relevance judge because cross-encoder reranking on ambiguous single-word queries lowerednDCG@10by 0.23 to 0.53 points. - Condition-constrained queries. When content conditions are present, Lune skips the relevance judge, which promoted false positives across 27 comparison queries and lowered precision from 0.929 to 0.908. Instead, a condition verifier extracts 350-character text windows around matches in each paper's own sections, tables, and abstract, up to 3,600 characters per paper, and evaluates up to 40 shortlist candidates in 4-paper batches. Lune returns only papers that satisfy every condition and places the verifying chunk first in the returned chunks. Across 21 condition-constrained queries, this verifier raised top-10 condition accuracy from 76.7 percent across 161 of 210 papers to 94.4 percent across 186 of 197 papers.
Server-side claim verification, gap diagnosis, and table extraction
Even when discovery tools locate the right papers, synthesis tasks require evaluating evidence across dozens of claims or papers at once, such as verifying a draft, diagnosing what evidence is still missing for a research question, or building a multi-paper comparison table. Running these tasks inside the client model requires fetching the full text of dozens of papers, which overflows the client model's context window and copies bulk text out of the paper corpus. Furthermore, when the client model summarizes multiple papers from long context, it misattributes claims across papers or alters quoted sentences.
Lune solves both problems by executing multi-paper claim verification, gap diagnosis, and table extraction on the server. This server-side synthesis evaluates up to 25 claims, 25 query angles, or 50 papers per call while enforcing exact-substring quote verification in Python and transferring zero raw full text to the client. Every server-side synthesis call enforces JSON Schema validation with inline model reasoning disabled:
- Chunk-attributed claim verification via
verify_claims. The client model submits up to 25 individual, testable claims in one call. For each claim, Lune retrieves the top 5 papers using multi-lane retrieval and cross-encoder reranking and includes each paper's abstract alongside matched chunks so the verifier never evaluates against empty context. A server-side language model classifies each claim as supported, unsupported, or lacking sufficient evidence, paired with supporting paper identifiers and a verbatim quote. A deterministic check in Python then verifies that every cited paper identifier belongs to the retrieved candidate set and that the quote is an exact, whitespace-normalized substring of a surviving candidate's chunk or abstract. If a quote fails this check, Lune downgrades the verdict to insufficient evidence and strips the unverified quote and citation before returning the response (Figure 5). - Automated gap diagnosis via
gather_evidence. Given a research task, up to 25 query angles, and an optional draft, a server-side language model decomposes the task into up to 8 individual evidence requirements. Lune runs parallel searches across the query angles and grades each requirement as covered, partially covered, or missing using substring-verified quotes. For uncovered requirements, Lune outputs targeted follow-up queries that isolate the missing evidence and, when configured for multi-step execution, runs bounded follow-up search loops on the server. - Structured table extraction via
extract_from_papers. Given up to 50 internal paper identifiers and typed column definitions of typestring,number,boolean, orstring[], Lune compiles a request-specific JSON Schema, reads up to 32,000 characters of chunks per paper filtered to requested sections such asResults, and extracts one validated row per paper in parallel without transmitting the raw full text.
… models which combine pre-trained parametric and non-parametric memory for language generation. We introduce RAG models where the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. We compare two RAG formulations, …
By combining protocol-guided tool orchestration, context-preserving full-text indexing, seven-lane retrieval with condition verification, and substring-verified server-side synthesis, Lune enables a client model to answer specialized scientific questions and diagnose missing evidence at inference time. Across benchmark evaluations, this architecture achieves 0.962 nDCG@10 on unconstrained queries and 94.4 percent top-10 accuracy on condition-constrained queries without fine-tuning model weights or surrendering custody of the paper corpus.
