Accelerated Distance-adaptive Methods for Hölder Smooth and Convex Optimization

This paper introduces new parameter-free first-order methods for convex optimization problems in which the objective function exhibits Hölder smoothness. Inspired by the recently proposed distance-over-gradient (DOG) technique, we propose an accelerated distance-adaptive method which achieves optimal anytime convergence rates for Hölder smooth problems without requiring prior knowledge of smoothness parameters or explicit parameter tuning. Importantly, our parameter-free approach removes the necessity of specifying target accuracy in advance, addressing a limitation found in the universal fast gradient methods (Nesterov, Yu. Mathematical Programming, 2015). For convex stochastic optimization, we further present a parameter-free accelerated method that eliminates the need for line-search procedures. Preliminary experimental results highlight the effectiveness of our approach on convex nonsmooth problems and its advantages over existing parameter-free or accelerated methods.