Distributional Reinforcement Learning with Monotonic Splines

One key challenge in quantile based distributional RL lies in how to parameterize the quantile function when minimizing the Wasserstein metric of temporal differences. Existing algorithms use step functions or piece-wise linear functions. We propose to learn smooth continuous quantile functions represented by monotonic rationalquadratic splines