Lune

EMNLP Findings2024

MP-RNA: Unleashing Multi-species RNA Foundation Model via Calibrated Secondary Structure Prediction

Heng Yang, Ke Li

Abstract

RNA foundation models (FMs) have been extensively used to interpret genomic sequences and address a wide range of in-silico genomic tasks. However, current RNA FMs often overlook the incorporation of secondary structures in the pretraining of FMs, which impedes the effectiveness in various genomic tasks. To address this problem, we leverage filtered highfidelity structure annotations for structure pretraining to enhance the modeling ability of FMs in single nucleotide resolution tasks. Experimental evaluations across four comprehensive genomic benchmarks demonstrate that our FM (MP-RNA) consistently outperforms existing RNA FMs, achieving a 40% improvement in RNA secondary structure prediction and obtaining top-tier results on DNA genomic benchmarks even though it has not been pretrained on any DNA genome. We release the code and tutorials 1 and models to encourage further research to bridge the gap between in-silico predictions and biological reality.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Understanding Transfer Learning of RNA Foundation Models on Downstream TasksYuan Li, Heng Yang, Renzhi Chen, Ke LiICML 2026

Builds on3

  1. HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide ResolutionEric Nguyen, Michael Poli, Marjan Faizi, Armin W. Thomas et al.NeurIPS 2023 · 574 citations
  2. Scaling Data-Constrained Language ModelsNiklas Muennighoff, Alexander M. Rush, Boaz Barak, Teven Le Scao et al.NeurIPS 2023 · 475 citations
  3. Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence ModelingYair Schiff, Chia-Hsiang Kao, Aaron Gokaslan, Tri Dao et al.ICML 2024 · 195 citations

Related papers

  1. Bridging Sequence-Structure Alignment in RNA Foundation ModelsHeng Yang, Renzhi Chen, Ke LiAAAI 2025
  2. HELM: Hierarchical Encoding for mRNA Language ModelingMehdi Yazdani-Jahromi, Mangal Prakash, Tommaso Mansi, Artem Moskalev et al.ICLR 2025
  3. Beyond Sequence: Impact of Geometric Context for RNA Property PredictionJunjie Xu, Artem Moskalev, Tommaso Mansi, Mangal Prakash et al.ICLR 2025
  4. PriFold: Biological Priors Improve RNA Secondary Structure PredictionsChenchen Yang, Hao Wu, Tao Shen, Kai Zou et al.AAAI 2025
  5. BioToken and BioFM – Biologically-Informed Tokenization Enables Accurate and Efficient Genomic Foundation ModelsAleksandr Medvedev, Karthik Viswanathan, Praveenkumar Kanithi, Kirill Vishniakov et al.ICML 2026
Dusk over the sea between two cliffs drawn in fine vertical lines