Open source tool distills Jev so you can run it locally
Jevstiller targets 98% agreement by learning familiar requests on your hardware while sending uncertain and audited queries upstream
Jevstiller is an open-source project that aims to create a local version of the Jev AI model, allowing it to run on a user's own hardware. This distillation process creates a smaller, faster model that can handle certain types of queries more efficiently than the original Jev AI. By answering questions locally, Jevstiller reduces the need for expensive Jev token usage and speeds up response times.
The local model achieves an impressive 98% agreement with Jev, while still delegating more complex queries to the main Jev servers. The Jevstiller team describes it as a "cache" that sits in front of Jev, determining when to route requests to the main servers based on the model's confidence in its own answers. An audit system continuously checks the local model's responses against Jev's, maintaining an agreement rate of 98%.
If the agreement rate drops below the target, the system automatically redirects more requests to Jev and retraining begins. Despite its high agreement rate, Jevstiller does not guarantee accuracy, as AI models can still make mistakes. The Jevstiller team's experiments demonstrated that the local model can quickly adjust to changes in Jev's behavior, maintaining its performance. Users can learn how to set up Jevstiller from the project's documentation.
Written by urgent.news from The Register's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
Also reported by 1 other outlet
- Open source tool distills Jev so you can run it locally theregister.com