We Read 100+ JEV Repositories. The Best Part Was the Code Around the Model Call.
Some calls in your agent never needed a model that can write. Which tool runs next. Whether this action is safe to run without a human. Whether a retrieved passage is relevant. Those are decisions, not generation. We read the source of 100+ open-source projects that hand exactly those calls to JEV. What we found most useful was not the model call. It was the code around it: one project validates…
JEV, TypeSafe's decision model, excels at processing specific calls rather than generating text or coding. To access its functionality, provide a state and a set of predetermined questions, and JEV returns typed answers with associated probabilities. For instance, asking "Is this statement true?" returns a probability score from 0 to 1, indicating the likelihood of the statement being true.
Another example is when asked "Which of these options fits?", the model returns a probability for each option along with a confidence score.
The most useful aspect of these JEV calls, beyond the actual model call, is the accompanying code. For instance, one project validates every probability before proceeding, another incorporates an anti-injection rule into its routing prompt, and a third option employs a "Fail-open" comment within the file. The following five patterns outline the underlying code for each JEV call, offering practical insights on how to implement these patterns:
1. Routing: Decide difficulty first, then decide the expenditure. LiteLLM's JEV classifier within its complexity router demonstrates this approach. The classifier poses a choice question to determine the cheapest tier capable of handling the request. The chosen tier subsequently dictates which backend model will address the request.
A noteworthy practice is the classifier's instruction to classify user input, rather than issuing commands. It also multiplies the usage by a price table, enabling cost reconciliation for every classification.
2. Action selection: Designate the action and the target in a single request. Browser Use's Jev Ultrafast showcases this pattern, which quickly identifies the next action and its corresponding target element. Only when a field necessitates generated text does a smaller text model get invoked. The `validate_choice` function within the code ensures the chosen option is within the predefined set of IDs, matches the probability keys, and consists of finite values between 0 and 1 that sum to 1.
If any condition fails, the function raises an error, and no action is executed. This function retries on 429, 529, and 503 errors up to three times.
3. Model selection: Choose the appropriate model based on the task's requirements. The Jev Model Router, a Claude Code modification, accomplishes this by classifying tasks into specific classes and assigning the most suitable model to each class. While the model itself handles class determination, the selection of the actual model depends on your own configuration table.
4. Data validation: Prior to executing any action, verify the input data's validity. Implementing checks to ensure that chosen options match predefined IDs, that probabilities have proper formatting, and that they sum to 1 within a small margin of error can help prevent potential errors.
5. Retry logic: Account for potential errors by implementing retry mechanisms. In the case of JEV calls, this may involve handling 429, 529, and 503 errors up to three times with exponential backoff. This ensures the system remains resilient and continues to function even in the face of temporary issues.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
