UniLG: A Unified Structure-aware Framework for Lyrics Generation

As a special task of natural language generation, conditional lyrics generation needs to consider the structure of generated lyrics 1 and the relationship between lyrics and music. Due to various forms of conditions, a lyrics generation system is expected to generate lyrics conditioned on different signals, such as music scores, music audio, or partially-finished lyrics, etc. However, most of the previous works have ignored the musical attributes hidden behind the lyrics and the structure of the lyrics. Additionally, most works only handle limited lyrics generation conditions, such as lyrics generation based on music score or partial lyrics, they can not be easily extended to other generation conditions with the same framework. In this paper, we propose a unified structure-aware lyrics generation framework named UniLG. Specifically, we design compound templates that incorporate textual and musical information to improve structure modeling and unify the different lyrics generation conditions. Extensive experiments demonstrate the effectiveness of our framework. Both objective and subjective evaluations show significant improvements in generating structural lyrics. * *Corresponding Author. 1 The structure of lyrics in our work means that lyrics have the chorus and verse parts among the sentences. 𝑏 ! , 𝑏 " , … , 𝑏 # Beat Music Score Lyric Polished Lyric Music Score with Lyric Audio Audio with Lyric 爱真的需要勇气 来面对流言蜚语 ………… Lyric UniLG (Masked) Lyric Bar Beat Segment Intro-position