{
  "id": 626052,
  "title": "The Model Is a Variable: Compiling a Neural Network Into Your C++ Binary",
  "url": "https://urgent.news/2026/08/12/the-model-is-a-variable-compiling-a-neural-network-into-your-c-binary",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-12T03:26:46.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/eugeneo_17/the-model-is-a-variable-compiling-a-neural-network-into-your-c-binary-oom"
  },
  "original_language": "en",
  "account": "The article discusses how a neural network can be integrated directly into a C++ binary, without the need for an inference server or API calls. The author presents a demo called dots, which places a trained neural network within a browser tab to serve as an opponent. The model consists of 2,760,322 parameters packed as fp16 and uses a 128-channel residual trunk running eleven 3x3 convolutions across a 24x24 board for each position it evaluates.\n\nThe author chose C++ for its ease of integration and portability, as it allows the model to be integrated into the same repository, build, review, and test environment as the application. The design brief for UchenML is to create small, fast models that can be easily embedded in any environment, whether it's the browser tab, inside an app, or a cloud service. This approach enables heterogeneous, distributed intelligence.\n\nTo achieve this, UchenML is built using C++20 and Bazel, and tested on various platforms including Linux, macOS, Windows, and under Emscripten for WebAssembly. The compute backend is hand-written portable SIMD code that runs on AVX-512, NEON, and WASM SIMD without requiring architecture-specific branches in the library.\n\nThe model itself is defined as a constexpr variable and is passed to a ModelParameters object, which supplies the weights. Inference is performed by calling the model with the input and parameters, and an optional scratch space can be preallocated to make the forward pass allocation-free. The model is designed to be a pure function of its input and parameters, allowing for reuse of a context across calls in a thread-safe manner.",
  "summary": "Overview I strongly believe software frameworks cannot be designed in the abstract. UchenML came out of my experiments integrating machine learning into edge applications, and out of what I want to build next — distributed, heterogeneous intelligence. The first demo, dots , puts a trained neural network in your browser tab as your opponent. No inference server, no API call — a 418 KB WebAssembly…",
  "key_points": [
    "Neural network integrated directly into C++ binary without inference server.",
    "Dots demo places trained model in browser tab to serve as opponent.",
    "UchenML creates small, fast models for heterogeneous, distributed intelligence."
  ],
  "editors_take": "Compiling a neural network into a C++ binary enables seamless integration and portability, allowing small, fast models to be easily embedded in various environments, promoting heterogeneous, distributed intelligence.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}