SVL: Goal-Conditioned Reinforcement Learning as Survival Learning

Standard approaches to goal-conditioned reinforcement learning (GCRL) that rely on temporaldifference learning can be unstable and sampleinefficient due to bootstrapping. While recent work has explored contrastive and supervised formulations to improve stability, we present a probabilistic alternative, called survival value learning (SVL), that reframes GCRL as a survival learning problem by modeling the distribution of time-togoal from each state. This structured distributional Monte Carlo perspective yields a closedform identity that expresses the goal-conditioned value function as a discounted sum of survival probabilities, enabling value estimation via a hazard model trained via maximum likelihood on both event and right-censored trajectories. We introduce three practical value estimators, including finite-horizon truncation and two binned infinitehorizon approximations to capture long-horizon objectives. Experiments on offline GCRL benchmarks show that SVL combined with hierarchical actors matches or surpasses strong hierarchical TD and Monte Carlo baselines, excelling on complex, long-horizon tasks. Webpage and code