EverSpark Forge V2 — Building a Distributed AI OS

EverSpark Forge V2 is now available, marking a significant architectural transformation for the project. This release introduces a Distributed AI Operating System where Archon serves as the control layer, and various Forge modules can operate on registered nodes across different machines, GPUs, and regions. The fundamental concept is that AI capabilities should not be confined to a single machine, GPU, or location.

In earlier versions of EverSpark Forge, the system was more localized, which, while effective for demonstrating the basic idea, had limitations. Different AI workloads had varying hardware requirements, making it challenging to allocate resources efficiently. To address this, V2 shifts the focus away from treating the machine as the core of the system and instead centers the architecture around distributed Forge modules.

At the heart of the V2 architecture is Archon/Orchestrator, which acts as the control layer. Remote machines operate as Pods and register with the main system via a Tailscale-based virtual network. Once a node is registered, Forge modules can be deployed to that node. For example, Archon running on the main machine can deploy Concept Forge on one remote node, Image Forge on another GPU node, and Audio Forge on a different machine or region.

These modules no longer need to reside on the same physical system, allowing for more flexible and efficient resource utilization.

The workflow for a typical V2 node includes active remote node registration, Tailscale-based networking, remote Forge deployment, node health checks, bandwidth testing, and region-aware node usage. Bandwidth testing is particularly important for remote GPU nodes, as a high-performance GPU becomes less useful if the network connection is too slow. EverSpark Forge can test the network performance of a Pod before incorporating it into the distributed system.

Concept Forge, responsible for handling LLM interaction and task understanding, is designed to support various model backends and external APIs, rather than being dependent on a single provider. Image Forge, an independent image-generation capability, can now run on remote nodes instead of being tied to the main machine. Generated results can be accessed directly from the remote node by the WebUI, eliminating the need to transfer large image files through the main control machine.

Audio Forge, another new addition, integrates VoxCPM2 and supports independent text-to-speech generation. It can also run remotely, with generated results accessible from its node.

The current V2 system demonstrates a fully functional distributed control and deployment loop. This means the system can connect remote nodes, register them, test them, deploy Forge modules, run AI workloads remotely, and access generated results across the wider system. This foundation is essential for future developments, such as natural-language request processing, task decomposition, and automated Forge dispatch.

Looking ahead, the longer-term direction for EverSpark Forge involves transforming natural-language requests into a task list, which is then dispatched to the appropriate Forge modules based on the requirements of the task. While this higher-level orchestration layer is still in development, V2 establishes the distributed foundation necessary for these future enhancements.

The current distributed loop is now operational, and Jhin, the project's creator, has recorded a demo showcasing the current distributed workflow, including remote deployment, Image Forge, and Audio Forge. The demo is available at https://youtu.be/yhaUHmxdNIA?si=rtGutyw_0o5Po1IS. For the most up-to-date functionality and implementation details, the project's source code on GitHub is the definitive resource.