Progressor: A Perceptually Guided Reward Estimator with Self-Supervised Online Refinement

We present Progressor, a novel framework that learns a task-agnostic reward function from videos, enabling policy training through goal-conditioned reinforcement learning (RL) without manual supervision. Underlying this reward is an estimate of the distribution over task progress as a function of the current, initial, and goal observations that is learned in a self-supervised fashion. Crucially, Progressor refines rewards adversarially during online $R L$ training by pushing back predictions for out-of-distribution observations in order to mitigate distribution shift inherent in non-expert observations. Utilizing this progress prediction as a dense reward together with an adversarial pushback, we show that Progressor enables robots to learn complex behaviors without any external supervision. Pretrained on large-scale egocentric human video from EPICKITCHENS, Progressor requires no fine-tuning on indomain task-specific data for generalization to real-robot offline RL under noisy demonstrations, outperforming contemporary methods that provide dense visual reward for robotic learning. Our findings highlight the potential of Progressor for scalable robotic applications where direct action labels and task-specific rewards are not readily available.