Jeff – Jev-compatible 0.8B decision models, trained at home, ~30 ms

Jeff, a compact decision-making model, can now make rapid judgments with zero-shot accuracy. This tiny model, trained locally, can assess options in mere milliseconds, providing calibrated probabilities for each choice without generating text. The model's versatility extends to various domains, from support queues and user intents to moderation labels, voice commands, and game moves.

Despite its size, Jeff maintains impressive accuracy, often matching or exceeding that of larger models like Jev. Its simplicity allows seamless integration into local code. While fine-tuning on specific examples can enhance its performance further, the model's reasoning, though less nuanced than Jev's, still proves effective in many scenarios.

To demonstrate its capabilities, Jeff engaged in three games, completing each move in 29-49 milliseconds on an Apple M4 Max, outpacing Jev's 212 milliseconds per call over its API. The models' performance was measured on various benchmarks, with Jeff generally matching or exceeding large models in classification and grounding tasks, but still trailing in reasoning-heavy benchmarks.

The full training process, from synthetic data generation to model development, is documented in the provided scripts. Jeff's development is based on an open recipe, and its code is released under an MIT license, while the model weights adhere to the Apache 2.0 license.