Desert Ant Labs: local, fast models that run on device

Desert Ant Labs, a European AI lab, is launching a new frontier in on-device intelligence. Their mission is to create small, specialized models for audio, vision, and text that can process information instantly and without cost. These models are designed to be powerful enough to function on devices as basic as a five-year-old phone, making them faster and more efficient than cloud-based alternatives.

Currently, they have launched 18 models (12 stable and 6 in beta) that can be accessed through a single SDK for Swift, Kotlin, and JavaScript. Each model is tailored for a specific task, optimized for speed and quality, and can be implemented with just a few lines of code. The first models are free up to 100,000 monthly active devices, with no tokens or logins required.

The company's founders, who previously worked on a video app named Detail, noticed the limitations of relying on cloud APIs for features like auto-edit and audio enhancement. To overcome these challenges, they trained their own models, resulting in significant improvements in speed, quality, and cost-effectiveness. For instance, they developed Clear, a model that enhances audio with better and faster results than Dolby, and Voz, which transcribes audio 5 times faster than its predecessor, Claude Sonnet.

With these models, Detail 6, set to launch with iOS 27, will replace all cloud APIs with local models, running entirely on the device. They believe that every developer has a wishlist of on-device models they'd build if cost weren't a factor. In fact, NVIDIA's researchers found that 40 to 70% of calls to large models could be replaced by small, specialized models.

The shift to on-device intelligence is significant, with the potential to reduce the $450 billion spent on data centers annually. Moreover, devices today come with increasingly capable chips that are well-suited to these tasks. Desert Ant Labs aims to provide developers with the tools to build with local models, starting from the cerebellum level – fast, specialized models for everyday tasks – and gradually moving up to more complex models when necessary.

Their goal is to make local models the default choice for developers, improving the developer experience and enabling them to create innovative products.