Emergent Dexterity Via Diverse Resets and Large-Scale Reinforcement Learning

Reinforcement learning in massively parallel physics simulations has driven major progress in sim-to-real robot learning. However, current approaches remain brittle and task-specific, relying on extensive per-task engineering to design rewards, curricula, and demonstrations. Even with this engineering, typical reinforcement learning methods can often fail on long-horizon, contact-rich manipulation tasks and do not meaningfully scale with compute, as performance quickly saturates when training revisits the same narrow regions of state space. We introduce OmniReset, a simple and scalable framework that enables on-policy reinforcement learning to robustly solve a broad class of dexterous manipulation tasks using fixed algorithm hyperparameters, no curricula, minimal reward engineering, and no human demonstrations. Our key insight is that long-horizon exploration can be dramatically simplified by using simulator resets to systematically expose the RL algorithm to the diverse set of robot-object interactions that underlie dexterous manipulation. OmniReset programmatically generates such resets with minimal human input, converting additional compute directly into broader behavioral coverage and continued performance gains for dynamic policies. We show that OmniReset gracefully scales to long-horizon dexterous manipulation tasks beyond the capabilities of existing approaches and is able to learn robust policies demonstrating a variety of dynamic, contact-rich recovery behavior. Finally, we distill OmniReset into visuomotor policies that can be transferred to the real world zero-shot, displaying robust retrying behavior to accomplish complex, contact-rich tasks with non-trivial success rates. Project webpage: https://omnireset.github.io