Jev TypeSafe AI + LangChain + Vercel AI Gateway

jev is an AI system designed specifically for taking decisions based on inputs. Unlike other LLMs that generate text, jev evaluates a given state and returns a typed decision along with probabilities of each option. TypeSafe reports that jev can be up to 193.6 times faster and 444.6 times cheaper than reference models. This article demonstrates how to use jev with LangChain through Vercel AI Gateway.

To integrate jev into an agent, install the skill jev-ai-gateway-langchain. The skill includes steps to configure AI Gateway and provide examples of primitives and middlewares. After installation, you can ask the agent to configure the connection, create classifiers with appropriate primitives, or incorporate middlewares into your LangChain application.

To use jev, you need to understand its input and output structure. A request to jev consists of two main parts: State, which are the data you want to evaluate (like a ticket, document, record, or agent history), and Questions, which are the decisions jev needs to make about that data. The format is state + questions → jev → typed answers + probabilities.

You can send multiple questions about the same state, and jev will evaluate them in parallel, returning each response under the identifier you define. Questions within a single request are independent. If a decision depends on another's result, you must make a second request. There are five key limitations to be aware of: Choice options have a maximum of 255; Score scales range between 2 and 10; Noul responses are binary with a probability between 0 and 1; The total context is 64k tokens for state + question; and the maximum input size is 32k tokens.