DNT: Diploid Genomic Foundation Model

A new reference-aligned diploid encoding for genetic variants has been introduced to enhance genomic language models. This encoding captures essential elements of diploid genotypes, such as zygosity, allele dosage, and the arrangement of multiple variants on homologous chromosomes. Traditionally, genomic language models process haploid sequences or treat haplotypes separately, resulting in a lack of direct representation of the paired genotype in a single sequence.

To address this limitation, researchers have developed unphased and phase-retaining tokenizers that can accept phased genotypes and convert them into a single-sequence diploid representation. The Nucleotide Transformer v3 backbones were utilized to continue training 8-million- and 100-million-parameter models. An auxiliary Contrastive Phase Loss (CPL) was designed to retain the phasing information of the variants within contextual representations.

The effectiveness of these models was evaluated on a novel compound-heterozygous benchmark containing 9,460 examples. Models that did not distinguish relative phase in their inputs performed near chance levels. However, the diploidic models demonstrated significant improvement in discrimination, with an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.649, compared to 0.506 for a vocabulary-adapted control model.

This finding establishes a method for making diploid genotype information accessible to genomic language models.

It is important to note that while these findings establish a method for incorporating diploid genotype information into genomic language models, further validation in naturally observed, accurately phased clinical cohorts is necessary to determine the universal improvement in variant prediction.