PrismML hopes its tiny LLM will change how we all use AI

PrismML, a small AI lab, has made a big splash in the industry with its development of tiny language models (LLMs) that can run on personal devices. Unlike other companies, PrismML hasn't raised significant funds yet, but the team behind it, led by Caltech professor Babak Hassibi, boasts impressive credentials and a unique technology aimed at making LLMs more accessible and efficient.

PrismML's latest release, Bonsai 2 27B, is a compressed version of Qwen3.8 27B, an open-source model developed by Alibaba. By using ternary weights—values of +1, -1, or 0—the model size has been reduced from 27 billion parameters to just 5.9 GB, a nine to ten times decrease in memory usage. This compression has only marginally affected the model's performance, retaining 98% of Qwen's benchmark scores — up from 95% in the previous version.

The startup's founder and leader, Hassibi, is optimistic about the future of larger models. He anticipates that as models grow, the compression technique will become even more effective, potentially reaching 100% benchmark performance parity with uncompressed models. Key advisor Ion Stoica, who co-founded Databricks and has ties to Berkeley's Sky Computing Lab, supports the project, emphasizing the privacy and convenience of running advanced models directly on users' devices, eliminating the need for cloud-based processing.