Lyrics Generation supported by Pre-trained Models

Abstract

Advancements in neural network architectures have improved the quality of several tasks in computational linguistics. Among the tasks benefited we can mention question and answer systems, dialogue systems, opinion mining and the automatic generation of texts, just to mention a few. Despite the advances, there is still room for contributions, since there are still open problems. In the case of text generation, especially in the musical genre, there are challenges for the production of texts that involve poetry and idioms. In particular, some of these challenges are linked to the treatment of metaphors and metonymy and the generation of paraphrases. This paper presents an analysis of the generation of excerpts of lyrics based on a pre-trained GPT-2 neural network model, after fine-tuning with two lyrics corpora, one in English and one in Portuguese. An analysis of the spelling, syntax and semantics of the generated texts are presented, as well as the discussion about the attempt to find a pattern in the sections generated by the implemented tool. Research demonstrates the potential for using such models in the generation of poetic texts.