Safe and Efficient: A Primal-Dual Method for Offline Convex CMDPs under Partial Data Coverage

Offline safe reinforcement learning (RL) aims to find an optimal policy using a pre-collected dataset when data collection is impractical or risky. We propose a novel linear programming (LP) based primal-dual algorithm for convex MDPs that incorporates “uncertainty” parameters to improve data efficiency while requiring only partial data coverage assumption. Our theoretical results achieve a sample complexity of O (1 / (1 − γ ) √ n ) under general function approximation, improving the current state-of-the-art by a factor of 1 / (1 − γ ) , where n is the number of data samples in an offline dataset, and γ is the discount factor. The numerical experiments validate our theoretical findings, demonstrating the practical efficacy of our approach in achieving improved safety and learning efficiency in safe offline settings.