SERA: Soft-Verified Efficient Repository Agents

Open-weight coding agents should hold a fundamental advantage over closed-source systems: they can be specialized to private codebases, encoding repository-specific information directly in their weights. Yet the cost and complexity of training has kept this advantage theoretical. We show it is now practical. We present Soft-Verified Efficient Repository Agents (SERA), an efficient method for training coding agents that enables the rapid and cheap creation of agents specialized to private codebases. Using Soft-Verified Generation (SVG), we generate thousands of trajectories from any code repository without requiring unit tests-enabling specialization to any downstream codebase. Beyond repository specialization, we apply SVG to a larger corpus of codebases, generating 200,000+ synthetic trajectories. Using only supervised finetuning (SFT), SERA achieves state-of-the-art results among fully open-source (open data, method, code) models while matching the performance of open-weight models like Devstral-Small-2. Creating SERA models is 26x cheaper than reinforcement learning and 57x cheaper than previous synthetic data methods to reach equivalent performance. SVG is built on two observations that emerged from simplification of previous methods: First, soft verification, where instead of testing the correctness of synthetic coding data via unit tests, we only compare the partial line-by-line overlap of patches generated from two rollouts. This removes the need for test infrastructure and enables data generation from any repository, practically removing limits on the amount of data we can generate from a single codebase as well as what codebases can be used. Second, vague instructions can diversify training data, increasing the proportion of data focused on non-bug related changes like refactoring. We find that these vague instructions improve SWE-bench performance as well as bug-focused data. In more detail, SVG is based on two rollouts from an agent: in the first, a teacher model is prompted with a vague instruction to make a change to a codebase starting from a randomly selected function, producing a trajectory and patch. This trajectory is converted into a synthetic pull request. In the second, the teacher model attempts to reproduce the patch given only the pull request description. Soft verification compares the two patches using line-level recall for training data selection. Taken together, this creates a cheap pipeline for high-quality data that enables rapid experimentation. We show through power scaling curves that private codebase specialization is highly sample efficient and matches or exceeds teacher model performance at low costs. Finally, we use our data to provide detailed analysis of scaling laws, ablations, and confounding factors for training coding agents. Overall, we believe our work will greatly accelerate research on open coding agents and showcase the advantage of open-source models that can specialize to private codebases. We release SERA as the first model in Ai2's Open Coding Agents series, along with all our code, data, and Claude Code integration to support the research community.