{
  "id": 279042,
  "title": "The next age of LLMs? Dev gets a small LLM running at 10 tokens a second locally on a $10 microcontroller",
  "url": "https://urgent.news/2026/08/07/the-next-age-of-llms-dev-gets-a-small-llm-running-at-10-tokens-a",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-07T19:35:00.000Z",
  "source": {
    "name": "TechRadar",
    "slug": "techradar",
    "url": "https://www.techradar.com/pro/the-next-age-of-llms-dev-gets-a-small-llm-running-at-10-tokens-a-second-locally-on-a-usd10-microcontroller"
  },
  "original_language": "en",
  "account": "A developer named slvDev has successfully run a 28.9-million-parameter language model locally at 9.88 tokens per second on a $10 ESP32-S3 microcontroller, according to recent wire material. The project, esp32-ai, was made available on GitHub under an MIT license in late July 2026 and has since gained over 3,600 stars and more than 470 forks of the underlying code.\n\nTo make the model run on such a resource-constrained device, slvDev had to scale down the model due to the limitations of the hardware. Despite this, the model still managed to generate text similar to TinyStories at this speed. The process involved two main steps: quantization and offloading.\n\nQuantization reduced the weights of the model from 16-bit precision to four-bit precision, significantly reducing the memory required. This brought the RAM requirement from nearly 60MB down to 14.9MB. However, the model still needed a large amount of memory to store its parameters, especially the embedding tables that are read rather than computed.\n\nTo further optimize memory usage, slvDev offloaded most of the model's parameters into the microcontroller's 16MB flash memory. This reduced the memory required for working memory to around 2MB. The remaining parameters are stored in the flash memory and accessed sparingly, which does not become a bottleneck.\n\nThe resulting performance of 9.88 tokens per second is faster than most people read. However, the model's capabilities are limited. It can generate short, mostly coherent stories but cannot answer questions, follow instructions, write code, or know facts. This is due to the small part of the model that performs the reasoning. The developer acknowledges that this approach does not make small models smarter but allows them to run on devices they previously couldn't. The second model, Barista, is designed to answer espresso-related questions.",
  "summary": "A 28.9M-parameter model runs on a microcontroller costing less than $10 at 9.88 tokens a second because 25M of those parameters never leave flash storage",
  "key_points": [
    "Developer slvDev runs 28.9M-parameter LLM locally on $10 ESP32-S3 microcontroller",
    "Quantization and offloading reduce memory requirements to 14.9MB and 2MB",
    "Model generates short, coherent stories but lacks advanced reasoning capabilities"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}