Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents

Magnitude, an open-source inference engine for agents, has been released by Y Combinator startup S25. This innovative tool optimizes itself to run open models as fast as possible on any hardware available, including Apple Silicon, NVIDIA, AMD, or even just a CPU. By compiling and tuning its kernels on the user's device, Magnitude enables open models to run up to twice as fast as llama.cpp.

Upon installation, users can connect their preferred agent, such as Pi, OpenCode, Hermes, Codex, or any other compatible agent, with a single click. The desktop app also includes the Magnitude Command Line Interface (CLI), eliminating the need for separate installations. As an open-source project, Magnitude is eager to grow its community; users can show their support by starring the repository.

The desktop application ships with precompiled kernels for a wide range of hardware classes, ensuring optimal performance without the need for extensive customization. Magnitude's kernels are specifically optimized for the most popular open-weight families, giving it an edge over generalist engines. The framework supports a variety of models, from smaller ones compatible with limited memory devices to larger models that benefit from additional memory resources.

Users can expect improved performance on any Apple Silicon, NVIDIA, or AMD GPU, or even without a dedicated GPU. Magnitude's platform allows prompts, files, and models to remain on the user's machine, ensuring privacy and eliminating the need for internet connectivity once a model has been downloaded.