{
  "id": 20077913,
  "title": "My Local AI Agent Needed a Bigger Brain, So I Gave It a Decentralized GPU",
  "url": "https://urgent.news/2026/11/14/my-local-ai-agent-needed-a-bigger-brain-so-i-gave-it-a-decentralized",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-14T18:00:02.000Z",
  "source": {
    "name": "HackerNoon",
    "slug": "hackernoon",
    "url": "https://hackernoon.com/my-local-ai-agent-needed-a-bigger-brain-so-i-gave-it-a-decentralized-gpu?source=rss"
  },
  "original_language": "en",
  "account": "This is the second progress update for the Decentralize AI Hackathon by HackerNoon and Nosana. The author has built a Local AI Linux Assistant called zkzkAgent that doesn't rely on the cloud, running entirely on their own hardware with Ollama. However, running everything locally has its limits. The challenge lies in the size of the models that fit on a laptop.\n\nzkzkAgent doesn't push every request through a single large prompt. Instead, it uses a small classifier node to determine the type of request: a question to answer, a single action to run, or a task that requires a plan. Plans are where the agent is most useful, and they demand the most from the model. On the author's laptop, a 2B model (qwen3.5:2b) fits in memory and answers quickly. However, a 27B model does not fit due to memory constraints.\n\nNosana introduces a third option: rent a GPU from a decentralized network, utilize it solely for the model, while keeping all other components at home. The article explains how zkzkAgent works in 30 seconds, detailing its multi-node LangGraph graph, its ability to classify requests, write plans, execute tools, and maintain a record of decisions with reasons.\n\nThe design rule states that only the model should move to a remote GPU, with all other components, including the graph, routing, confirmation gates, and 25 tools, running on the user's machine. If the model suggests executing a destructive tool like remove_file, it must still be approved by the user through a chat interface.\n\nThe author outlines the design rule, stating that Nosana gets the model calls and nothing else. The graph, routing, confirmation gates, and all 25 tools continue to run on the user's machine. The model on a remote GPU can suggest running remove_file, but only on the user's machine, after the user confirms with a 'yes'. This rule limits the change to a single file, models/LLM.py, and a single setting in the `.env` file.\n\nThe author details how to integrate Nosana's inference API with the agent using LangChain's `ChatOpenAI` client, allowing the agent to communicate with Nosana's API. The rest of the graph remains unaware of the backend, treating the LLM as a simple object.\n\nTo test the integration, the author switched to the 27B model on Nosana and tasked zkzkAgent with creating a new project called zkzk_ai, a calculator web app with a FastAPI backend and an HTML, CSS, and JavaScript frontend. The author then walked through each step of the process, from routing and planning to executing the tools.",
  "summary": "Running everything locally has a ceiling, and I kept hitting it. Here's how I fixed it.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}