FG-Midiformer: A Symbolic Music Understanding Model towards Fine-Grained Learning of Multi-Attributes

Symbolic music understanding is a fundamental task in multimedia interpretation, which aims to decode musical attributes from symbolic music representations. Existing methods usually handle musical sequences as linguistic data, ignoring intrinsic musical properties. For instance, recurring melodies often appear in musical pieces with subtle variations, which requires methods that are aware of both local musical details and overall repetitive patterns, yet current methods can not fit the request. To address this issue, we introduce FG-Midiformer, a modified transformer framework that incorporates a multi-scale-aware feature learning (MSAFL) module and a local feature enhanced classification (LFEC) module for fine-grained understanding of multi-attributes. Specifically, the MSAFL module is designed to capture multi-scale musical relationships by embedding efficient multi-scale attention for long-term dependency modeling. In order to improve the classification accuracy of musical attributes, we devise the LFEC module, in which an attention mechanism with full 3-D weights is first introduced to efficiently highlight and leverage important local musical features. The LFEC module strengthens local feature representation and improves the sensitivity of FG-Midiformer to subtle differences between musical attributes. Extensive experiments show that FG-Midiformer achieves state-of-the-art performance in multi-attribute understanding tasks such as melody identification, velocity prediction, composer categorization, and emotion classification. The code will be released at https://github.com/Viki66666/FG-Midiformer.