Learning Value Functions in Deep Policy Gradients using Residual Variance

Stochastic policy π θ (a|s) with parameter θ An agent in state s t interacts with an environment by sampling action a t ∼ π θ (•|s t ), receives reward r t and transitions to a new state s t+1 .