Jev vs LLMs: Why AI Agents May Need a Decision Layer
Jev vs LLMs: Why AI Agents May Need a Decision Layer Here's an uncomfortable pattern in modern AI applications: User input ↓ LLM ↓ generated text ↓ parser ↓ application logic ↓ action We're often using a general-purpose language model to make a tiny decision. Should we retry? Should we escalate? Which tool should we call? Which model should handle this? Should this request be blocked? Those are…
A growing issue in AI applications is the reliance on general-purpose language models to make minor decisions. These questions, such as whether to retry, escalate, call a tool, choose a model, or block a request, are not generation problems but decision problems. TypeSafe AI's System One model, Jev, offers a unique solution to this challenge.
Jev is designed around structured decisions rather than open-ended string generation. Its core idea is to convert the state of the application into a typed decision, rather than generating a string. This is in contrast to traditional LLMs, which produce a generated string from a given state.
The key difference between Jev and LLMs lies in their outputs. LLMs provide a choice, score, or control flow, which the application then needs to interpret and act upon. This involves prompt instructions, generated output schema adherence, parsing, interpretation, and potentially unnecessary text generation. In contrast, Jev directly provides a decision, such as which team should handle a request, whether it's urgent, or how frustrated the customer is.
A practical agent architecture using Jev involves proposing a tool call, which is then decided upon by Jev. The decision is then consumed by the application code, which controls authorization, thresholds, audit logs, and side effects. This ensures that the deterministic application layer remains in control, while Jev handles the decision-making process.
The probabilities returned by Jev are crucial. They allow the application to decide whether to execute a tool call, request human approval, or block the tool call based on its own risk tolerance. This makes Jev a component within a larger control system, rather than the system itself.
Jev's output types include Choice, Score, and Noul. Choice is used when a decision between a few options is needed, such as which model should process a request. Score is used when a magnitude needs to be determined, such as the urgency of a request. Noul is used when a numerical value needs to be provided, such as a frustration score.
In conclusion, Jev provides a structured approach to decision-making in AI applications, allowing developers to define the decisions their application needs to make and ensuring that the application code remains in control. This approach offers a more efficient and reliable way to handle decision problems compared to the traditional LLM approach.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
