Calibration-free compression brings Evo 2 to its full million-token context on a single GPU

Evo 2, the largest openly available genomic foundation model, features 40 billion parameters that exceed the capacity of a single 80 GB accelerator for genome-scale analysis. To overcome this limitation, researchers have introduced TurboQuant-Bio, an open toolkit that compresses Evo 2's weights and attention cache to four bits, employing calibration-free compression without requiring calibration data.

This compression is near-lossless across various applications, including perplexity spanning the tree of life, genomic classification, splice-site prediction, gene completion, and clinically relevant variant-effect prediction.

With the aid of TurboQuant-Bio, Evo 2 40B can now run on a single 80 GB GPU, while Evo 2 7B can achieve its full million-token context within a 40 GB memory budget. This represents an eightfold increase in the reachable context for Evo 2. However, the researchers also identified an issue with the released chunked-prefill path, which returns plausible but uncorrelated likelihoods.

To rectify this, they derived a block-wise continuation that accurately repairs the problem, enabling a complete 580-kilobase bacterial genome to be scored within a single context in just 22 minutes. In contrast, the previous method took 13.7 hours to accomplish this task.