Open source tool distills Jev so you can run it locally
Jevstiller targets 98% agreement by learning familiar requests on your hardware while sending uncertain and audited queries upstream
Jevstiller is an open source project that distills the outputs of Jev, an AI tool that answers questions in a cheap and quick manner, into a local model. This local model can handle some of Jev's requests more efficiently, reducing the need for expensive token usage and faster response times. The model constructed by Jevstiller is small and fast, and if successful, can achieve 98 percent overall agreement with Jev.
Jevstiller works by first learning from Jev's responses to various queries, and then using this knowledge to answer similar questions locally. If it's confident in its ability to answer a question, it does so without needing to send the request to Jev. However, if the local model is unsure or encounters a difficult question, it forwards the request to Jev.
The accuracy of Jevstiller's responses is not guaranteed, as it's still an AI model that can make mistakes. But by continuously auditing its responses against Jev's, Jevstiller aims to maintain a high level of agreement with the original AI tool.
Written by urgent.news from The Register Science's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
This story
This is one outlet's version. Read the fullest account.
- Open source tool distills Jev so you can run it locally theregister.com