Rediscovering client/server: The future of AI is distributed, not centralised

For the past year, the author has been utilizing AI to generate Python programs on an Ubuntu server. During this process, they had to learn Linux commands again, which was an eye-opening experience. The author was amazed by how swiftly AI could produce code, troubleshoot it, and present a functional solution. This newfound ease contrasted with the previous requirement of substantial programming expertise.

However, they also discovered the costly nature of AI token usage and the subtle attempts by platforms to persuade users to purchase more. From this experience, the author pondered if there was a more affordable and practical method of achieving the same results.

The author realized that repurposing an old Intel NUC could serve as a solution. By using cloud AI for Python program development and employing the NUC to manage scheduling, execution, and the delivery of daily reports, they could allocate expensive AI computation to tasks genuinely requiring it. While the NUC lacks a high-performance GPU, its CPU and 32 GB of memory are sufficient for executing automated workflows.

The author acknowledges that running demanding language-model inference locally would be challenging, but that is precisely the point: not every task necessitates a powerful AI model. They concluded that this rediscovered architecture echoes early enterprise computing principles: client/server. Tasks requiring less computational power can be executed locally, while more resource-intensive processes can be offloaded to more capable systems, such as cloud providers and specialized AI infrastructure operators like Neoclouds.

Recent developments reinforce this trend, albeit with different focuses. Meta's Muse Glimmer is an open-weight model engineered for local agentic workflows and accessible through Ollama. When appropriately paired with hardware and software, it can become a component of a private AI assistant designed to handle tasks on a user's behalf.

Meanwhile, Microsoft's Surface Laptop Ultra showcases how AI processing capabilities are migrating to personal devices. Certain configurations integrate a 20-core CPU with a Blackwell-based GPU featuring 6,144 CUDA cores, enabling the execution of compatible local models while still allowing access to more powerful cloud-based systems when necessary.

While these examples target distinct aspects—model and its agentic capabilities, versus the computing platform—they collectively indicate a future where AI will be distributed across devices, local servers, and remote infrastructure, rather than confined to a single hub.

Three distinct layers of pervasive AI will emerge: personal AI, business AI, and sovereign AI. Personal AI represents a private digital assistant capable of understanding an individual's routines, preferences, strengths, and weaknesses. This assistant could manage files, monitor household systems, oversee recurring tasks, and coordinate with specialized cloud models when needed.

The appeal of local control lies in privacy, cost-effectiveness, and the ability to keep sensitive information closer to home. Business AI encompasses AI-capable laptops, enterprise applications, and company-controlled systems that assist employees with mundane work. Routine processing may occur on the device or within a private environment, while more intensive analysis is delegated to cloud services.

The true opportunity lies not only in accelerated inference but also in seamlessly integrating AI into business processes with appropriate security, permissions, and human oversight.

Sovereign AI refers to governments increasingly seeking control over AI systems used in healthcare, transportation, emergency services, defense, and public administration. Sovereignty entails making decisions regarding data residency, infrastructure management, model governance, and the ability to audit and adapt systems. Open-weight models may be particularly valuable in this context, although they are not the sole solution.

These three layers—personal AI, business AI, and sovereign AI—are not isolated markets; rather, they form an interconnected ecosystem. A personal assistant may utilize a local model for everyday tasks, an enterprise system for work-related functions, and a frontier cloud model for complex problems. The key lies in directing each workload to the most appropriate layer.

With the proliferation of ubiquitous AI, governance becomes paramount. A system that aids in drafting emails poses different risks compared to one that coordinates emergency services or operates critical infrastructure. Capability alone is insufficient; accountability, transparency, safeguards, and human judgment must keep pace.

While the author expresses optimism about the future of AI, they caution against the creation of a system like Skynet, which held little regard for human life. Instead, they remain hopeful that R. Daneel Olivaw will protect us from such dangers.