Ditch the data center: I fired my cloud chatbot and built a free local AI agent to run on my PC — and you can too
Ditch the cloud subscription tax. Here is how I fired my cloud chatbot, built a free local AI agent on my PC with Hermes, and took back my data privacy.
In today's technological landscape, many individuals pay a monthly fee to rent server space in a data center for cloud AI services. While these services are fast and convenient, they put users' personal data and workflows at risk by handing over control to large tech companies. This creates a potential bubble in the AI market and exposes sensitive information to an ever-learning black box. However, there is a better alternative: building a local AI agent on your personal computer.
The future of AI is local, and recent advancements in hardware have made this possible. With the release of Nvidia RTX Spark laptops and Apple's upcoming M6 MacBook Pro models, there is now sufficient processing power to run AI agents without relying on the cloud.
An AI agent differs from a standard chatbot in that it has autonomy. Instead of waiting for a user to ask a question, an agentic AI takes on a goal, uses tools like a file system or web browser, and executes tasks independently through a ReAct (Reason + Act) loop. This provides greater control and privacy compared to cloud-based options.
To set up a local AI agent, Hermes Agent is a recommended open-source solution that runs on Windows and Mac. The installation process is straightforward, involving downloading the package, selecting app connectors, and choosing a color theme. Next, users must switch the provider setting to "local" and download a local model to act as the agent's brain.
The choice of model depends on the laptop's RAM. For laptops with 8-16GB RAM, smaller 7-8B parameter models like Llama 3.1 8B are suitable. Laptops with 32-64GB RAM can run 30B-70B parameter models like Qwen 3.8 27B, while those with 128GB+ RAM can utilize enterprise-grade models that rival GPT-4, all without internet access.
Quantization, a process that reduces the size of large models, is essential for running massive AI brains on consumer hardware. By quantizing models, the memory-hogging mathematical weights are rounded down, resulting in a drastic reduction in file size. This allows for efficient use of limited laptop resources.
Written by urgent.news from Tom's Guide's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
