Show HN: Strot – Compile an LLM's intent once, run decisions locally forever

Strot is a tool that allows developers to compile their intent once using a language model (LLM), and then execute decisions locally on their own hardware without any marginal costs. The LLM only runs once during the compilation process, and the production stage involves a frozen, self-contained Python artifact that serves as a decision model for every record. This approach ensures that every output is bounded, and low-confidence records abstain instead of guessing.

The core of Strot's functionality is the "run()" function, which routes each record to an action and executes the compiled handler. Additionally, the "decide()" function enables users to answer typed questions (choice, score, or boolean) per record. Strot utilizes the Julia-1 model from SupersonicLabs, which is a 144M parameter model that runs on CPU. The tool comes with clean Docker containers, and the entire process, including benchmarks and loss details, is well-documented.

Strot aims to simplify the deployment of AI models by baking the local model into your image for offline production. This eliminates the need for pip installations of unrelated packages like Julia. The developers have provided extensive documentation, including an architecture overview, the compile lifecycle, calibration semantics, deployment guidelines, model swapping instructions, and a full API reference.

The project is licensed under the MIT license, and the default decision model is Apache-2.0, which is associated with the SupersonicLabs/Julia-1 model. The tool is built upon the STROT (arXiv:2505.01636) framework, which focuses on compile-time LLM code synthesis with deterministic, self-routing natural language understanding (NLU) runtimes. Strot can be installed via PyPI using the package name "strot-ai".