Conditional Generation And Inpainting Of Non-coding RNA Sequences With Masked Discrete Diffusion
Designing functional non-coding RNA (ncRNA) is fundamental to synthetic biology and RNA therapeutics, yet generative modelling for ncRNA has received far less attention than protein design. We present RNA-MDLM, a framework that extends Masked Discrete Language Models (MDLM) to the conditional generation and inpainting of ncRNA. We make two additions: first, conditioning on RNA-type…
A new framework called RNA-MDLM has been introduced, which enhances Masked Discrete Language Models (MDLM) for generating and editing non-coding RNA (ncRNA) sequences. The framework incorporates two key enhancements: first, it conditions the model on RNA-type representations from a pretrained RNA language model, and second, it employs a modified classifier-free guidance scheme (Mod-CFG) to fine-tune the balance between conditional, unconditional, and random-sequence probabilities.
To evaluate the performance of RNA-MDLM, the researchers developed a benchmark called REPAINT GAMES, which consists of seven structured masking tasks designed to assess the model's ability to capture sequence patterns, structural motifs, and base-pairing patterns specific to different RNA types. The researchers trained the model on a vast dataset of 4.6 million ncRNA sequences, spanning six evaluable classes.
The results of extensive ablation studies reveal that the embedding-conditioned RNA-MDLM offers the optimal compromise between structural fidelity, biological novelty, and inpainting accuracy. Moreover, the model's performance in steering generation is significantly more influenced by class labels compared to a plain label baseline.
The researchers also discovered that a model trained on a smaller, class-balanced subset can appear more realistic due to the copious presence of natural sequences, rather than learning the underlying rules. To further validate the effectiveness of their approach, the model was compared against a masked-diffusion model and a family-specific VAE in the context of ribozyme families.
The findings suggest that a type-conditioned model, such as RNA-MDLM, and a per-family model address distinct tasks, which must be considered in fair comparisons.
The researchers have made their code and trained models publicly available to facilitate further research and development in the field of ncRNA generative modeling.
Written by urgent.news from bioRxiv's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
