Drop-in Circulant Structural Priors for Transformer Decoding of Cyclic Codes

While Transformer-based architectures have revolutionized neural decoding, existing models often treat codes as generic sequences, ignoring their inherent algebraic properties. In this paper, we take a step toward bridging these two domains by proposing a decoding approach that integrates the algebraic structure of cyclic codes into Transformer-based decoders. Building on coding theory, we introduce two key notions, error correction patterns and inter-node relationships, and show how they can be exploited in neural architectures. By further leveraging the inherent cyclic properties of these codes, we propose a plug-and-play, flexibly deployable decoding method tailored for cyclic codes, which links the structural characteristics of the codes to the model parameters. Experimental results show that our method reduces the bit error rate (BER) by about one order of magnitude on average, while also reducing the total number of parameters by approximately 97%. Additional comparative experiments provide evidence supporting our proposed notions and highlight a promising pathway for bridging classical coding theory and modern Transformer-based decoding architectures.