Large Displacement Motion Transfer with Unsupervised Anytime Interpolation

Motion transfer is to transfer pose in driving video to the object of the source image so that the object of the source image moves. Although great progress has been made recently in unsupervised motion transfer, many unsupervised methods still struggle to accurately model large displacement motions when large motion differences occur between source and driving images. To solve the problem, we propose an unsupervised anytime interpolation-based large displacement motion transfer method, which can generate a series of any time interpolated images between source and driving images. By decomposing large displacement motion into many small displacement motions, the difficulty of large displacement motion estimation is reduced. In the process, we design a selector to select optimal interpolated images from generated interpolated images for downstream tasks. Since there are no real images as labels in the interpolation process, we propose a bidirectional training strategy. Some constraints are added to the optimal interpolated image to generate a reasonable interpolated image. To encourage the network to create high-quality images, a pre-trained Vision Transformer model is used to design constraint losses. Finally, experiments show that compared with the large displacement motion between source and driving images, the small displacement motion between interpolated and driving images makes it easier to realize motion transfer. Compared with existing state-of-theart methods, our method significantly improves motion-related metrics.