Introducing Swarm: Multi-Agent Orchestration and an LLM Gateway in Pure Rust ๐ฆ
While experimenting with multi-agent systems, I kept ending up with two separate pieces of infrastructure: an orchestration layer for agents and tools, and a gateway layer for routing LLM requests. I wanted both to share the same runtime, provider abstractions, state management, and protocol contracts. So I built Swarm , an open-source AI orchestration framework and model gateway written in Rust.โฆ
Swarm is an open-source AI orchestration framework and model gateway written in Rust. It addresses the need for both an orchestration layer for agents and tools, as well as a gateway layer for routing LLM requests, by unifying them into a single high-performance Tokio runtime. Swarm offers two modes: Multi-Agent Orchestration Mode, which handles complex reasoning and multi-agent coordination, and Model Gateway Server Mode, which serves as an OpenAI-compatible gateway for client applications and automated pipelines.
The Multi-Agent Orchestration Mode includes specialized services such as a Planner Agent, which dynamically generates execution DAGs based on user requests, an Executor Agent that resolves task dependencies and controls step execution, Domain Specialists that execute live tools using a native Model Context Protocol (MCP) runtime, and Discovery & Memory services for maintaining service registries and conversational state. Evaluation and Judge Services are also included for output verification and self-correction.
In the Model Gateway Server Mode, Swarm exposes an OpenAI-compatible gateway that supports stateful responses, multi-turn conversation chaining, and unified multi-provider routing across various backends like Groq, Google Gemini, OpenAI, and local endpoints such as Ollama, vLLM, and llama.cpp. The gateway configuration is done through a TOML configuration file, allowing for easy integration with different models and providers.
Swarm is built using Rust, which provides several advantages for orchestration and gateway workloads. It offers low-overhead request handling with Tokio and Hyper, concurrent session management using DashMap and Arc-based stores, a small runtime footprint with no garbage collector, and strongly typed protocols to reduce schema mismatches and integration errors. The framework is fully open-source under the Apache-2.0 license and relies on the emerging Rust AI ecosystem, including the official MCP Rust SDK and A2A Protocol.
Written by urgent.news from Dev.to's reporting โ not their text. Machine-written โ may contain errors; check the original before relying on it.
