Evaluating the performance of splicing predictors on thousands of synthetic gene variants
Computational predictors of RNA splicing are increasingly used to interpret genetic variants and to design synthetic genes, yet they are almost always benchmarked on endogenous human sequences closely related to their training data. Whether their performance reflects genuine recognition of splicing signals, or instead exploits statistical features of natural genomes such as conservation and…
A recent study evaluated the performance of eleven different computational predictors in analyzing thousands of synthetic gene variants that are distinct from any training data. Using long-read sequencing, the researchers directly measured splicing at each position in the variants. Despite the significant distribution shift between the synthetic variants and the training data, modern deep-learning predictors maintained strong performance.
The ranking of the predictors remained largely stable when comparing position-level and construct-level benchmarks.
SpliceTransformer ranked highest, followed by AlphaGenome and SpliceAI. Predictors that do not consider long-range sequence context performed substantially worse, mainly because they assign high scores to many non-spliced positions. The findings agree with benchmarks on endogenous variants, suggesting that the leading models capture transferable, sequence-intrinsic determinants of splicing.
The study also introduced a unified calibration method that translates each predictor's scores into the measured fraction of spliced reads, enabling direct comparison and interpretation of scores as splicing outcomes. The results demonstrate that current deep-learning models generalize beyond natural genomes and offer a practical framework for splicing-aware sequence design.
Written by urgent.news from bioRxiv's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
