{
  "id": 18502104,
  "title": "I built a private, offline AI chat you can run with one command: npx unmuzzled",
  "url": "https://urgent.news/2026/11/06/i-built-a-private-offline-ai-chat-you-can-run-with-one-command-npx",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-06T13:09:47.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/rahul_agarwal18/i-built-a-private-offline-ai-chat-you-can-run-with-one-command-npx-unmuzzled-1bl"
  },
  "original_language": "en",
  "account": "Most AI chat applications send every user prompt to remote servers for processing. The creator sought a different approach: a chat application that runs the model on the user's own hardware, operates offline, and can be started with a single command. This chat application is named \"Unmuzzled\" and is open-source, licensed under the MIT license. This post explains how Unmuzzled works and discusses some unexpected challenges encountered during its development.\n\nWhen you run \"npx unmuzzled\" in your terminal, an interactive chat interface appears in your web browser. During the first run, a setup screen displays information about the selected model, its size, available RAM, and disk space. The application then proceeds to download the required model files only when prompted by the user.\n\nUnmuzzled runs the AI model locally using the llama.cpp library, which supports Vulkan, CUDA, or Metal acceleration when available. The chat responses are displayed in markdown format with syntax highlighting, and the model's reasoning is shown in a collapsible, timestamped panel for easy reference. Users can edit, regenerate, and search their conversation history within the browser.\n\nThe application only stores chats in the user's browser, ensuring privacy. It supports any GGUF model format, allowing users to specify their own model using the command \"npx unmuzzled --model ./your-model.gguf\". The underlying model used by Unmuzzled is a 9 billion parameter model, which is a modified version of Qwen3.5-9B. This model is fine-tuned using ~250,000 reasoning samples generated by DeepSeek-V4-Pro, with the refusal behavior removed using Heretic. The modified model is quantized using Q4_K_M, reducing its size to 5.63 GB while retaining its ability to generate accurate answers, although it is not guaranteed to be correct.\n\nUnmuzzled is built with minimal dependencies to ensure a lightweight and straightforward desktop experience. It does not include any framework or require a build step on the desktop side. The application is comprised of several key components: a command-line interface (bin/unmuzzled.js), an HTTP server (src/server.js) handling static files and streaming chat API (SSE), a wrapper for the node-llama-cpp library (src/engine.js), a model catalog (src/models.js), and the interface files (public/).\n\nThe streaming mechanism works by sending conversation data from the client to the server via POST requests at /api/chat, and the server responds with Server-Sent Events (SSE) that include thought chunks, text chunks, and a final \"done\" event containing token count and tokens-per-second metrics. To ensure secure communication, Unmuzzled requires application/json payloads and rejects any request with a Host header that is not a local IP address, protecting against both cross-site request forgery (CSRF) attacks and DNS rebinding attacks.\n\nOn the user's laptop (Intel i5-1135G7 with integrated graphics via Vulkan, 16 GB RAM), the 9B model runs at approximately 2 tokens per second (ranging from 1.7 to 2.7 tokens per second across various runs) after an initial load time of 40-50 seconds. Keep in mind that \"offline\" does not imply \"fast\"; the main limitation is the user's hardware capabilities. For faster performance, consider using a discrete GPU or an Apple Silicon Mac, both of which tend to provide better processing power.\n\nA beta version of the Unmuzzled application is also available for Android devices, which runs the same local model on the phone using the llama.rn library within a React Native environment. This Android version uses llama.cpp builds optimized for modern ARM cores and optional OpenCL offloading for Adreno GPUs. However, due to Google Play's AI-content policy, the application is not published there and can be downloaded from GitHub Releases instead.",
  "summary": "Most AI chat apps send every prompt to someone else's servers. I wanted the opposite: a chat app where the model runs on my hardware, works offline, and takes one command to start. npx unmuzzled That's Unmuzzled . It's open source (MIT), and this post covers how it works and the problems I didn't expect. A real run on my laptop. The reasoning streams first, then the answer. Generation is sped up…",
  "key_points": [
    "Unmuzzled runs AI model locally using llama.cpp library",
    "Chat responses displayed in markdown format with syntax highlighting",
    "Application stores chats in user's browser for privacy"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}