OpenAI makes its Jev competitor widely available as decision model competition heats up

OpenAI has made its Decisions API model widely available to users, joining the market for fast, affordable intelligence models. TypeSafe, a startup founded by a former OpenAI researcher, created a category of deep learning models known as decision models, which can make quick choices and provide confidence ratings for a set of options. Software developers quickly discovered that these decision models could replace large language models (LLMs) in various workflows, offering faster operation and lower costs.

OpenAI's product lead, Nikunj Handa, said his team was impressed by TypeSafe's Jev model and raced to develop their own version. Instead of building a specifically calibrated decision model, they repurposed ChatGPT Luna to speed up inference and reduce costs. The big breakthrough with Decisions API has been the speed, with OpenAI shrinking inference times from one second to as low as 120 milliseconds.

OpenAI plans to further optimize models by developing specific training methods. However, for now, they are using ChatGPT Luna for the Decisions API.

OpenAI added voice and visual modalities to the public version of the Decisions API and worked to reduce latency in different regions. Initial trials with companies like Box, Datadog, Ramp, and Speak showed that engineers replaced OpenAI's LLMs with the Decisions API and achieved higher throughput. For instance, Speak reduced the time between actions from 1.4 seconds to 0.13 seconds. OpenAI also used the product to enhance its Dots voice assistant's responsiveness.

However, it's unclear what constitutes a good decision model, as optimization across intelligence, cost, calibration, and speed varies depending on the workload. TypeSafe CEO Diogo Almeida doubts benchmarks and advises users to test the models themselves. TypeSafe released rankings for its latest Jev model, which includes support for images and smart-if statements, placing it seventh and third on two public benchmarks.

Despite OpenAI's Decisions Model being eighth and fourteenth on the same rankings, TypeSafe's offerings are currently the most widely-used on OpenRouter.

The most entertaining benchmark for Jev models is JevMan, where decision models play PacMan. The latest Jev model ranks first in this benchmark. OpenAI's Luna model may have an advantage in multi-modality for image classification, as it shows better calibrations. The speed of the Decisions API may enable new use cases, such as processing every frame in a video within milliseconds.