DeepWriteSYN: On-Line Handwriting Synthesis via Deep Short-Term Representations

This study proposes DeepWriteSYN, a novel on-line handwriting synthesis approach via deep short-term representations. It comprises two modules: i) an optional and interchangeable temporal segmentation, which divides the handwriting into short-time segments consisting of individual or multiple concatenated strokes; and ii) the on-line synthesis of those short-time handwriting segments, which is based on a sequence-to-sequence Variational Autoencoder (VAE). The main advantages of the proposed approach are that the synthesis is carried out in short-time segments (that can run from a character fraction to full characters) and that the VAE can be trained on a configurable handwriting dataset. These two properties give a lot of flexibility to our synthesiser, e.g., as shown in our experiments, DeepWriteSYN can generate realistic handwriting variations of a given handwritten structure corresponding to the natural variation within a given population or a given subject. These two cases are developed experimentally for individual digits and handwriting signatures, respectively, achieving in both cases remarkable results. Also, we provide experimental results for the task of on-line signature verification showing the high potential of Deep-WriteSYN to improve significantly one-shot learning scenarios. To the best of our knowledge, this is the first synthesis approach capable of generating realistic on-line handwriting in the short term (including handwritten signatures) via deep learning. This can be very useful as a module toward longterm realistic handwriting generation either completely synthetic or as natural variation of given handwriting samples. The availability of large-scale public databases, together with the remarkable progress of deep learning, have led to the achievement of very impressive results in many different fields. However, there are still applications and scenarios in which data are scarce, achieving in general poor results. These scenarios are usually known in the literature as zero-, one-, and few-shot learning (Wang et al. 2019 (Wang et al. , 2020)) . Different approaches have been proposed in response to the lack of data, being the synthesis of artificial data one of the main research lines followed nowadays. This is of special relevance in image-based scenarios due to the popular Generative Adversarial Networks (GAN) and Transfer Learning techniques (Goodfellow et al. 2014; Tan et al.