Building Multi-Agent Systems That Actually Scale: Lessons from Hermes, LobeHub, and the 2025 AI Agent Explosion

The AI agent landscape underwent a significant transformation in 2025. What started with single-agent chat interfaces evolved into complex multi-agent ecosystems involving dozens or even hundreds of specialized agents collaborating, debating, and executing intricate workflows. Two noteworthy examples, Hermes and LobeHub, demonstrated distinct architectures that shed light on the challenges and solutions for building scalable multi-agent systems.

The fundamental challenge of scaling multi-agent systems lies in the complexities introduced by communication, orchestration, and cost. While a single agent interacting with an LLM is straightforward, introducing multiple agents adds layers of difficulty. Agents must exchange messages, share state, and coordinate actions, resembling a distributed systems problem.

Orchestrating the execution path, deciding which agent should run when and under what conditions, adds another control plane layer. Moreover, every inter-agent message potentially triggers an additional LLM call, leading to a significant increase in cost and latency. The naive approach of firing all agents in parallel and later aggregating results can lead to resource contention, unbounded fan-out, and even prompt injection attacks.

Hermes adopted a message-passing system with typed interaction protocols to address these issues. The key insight for Hermes was to make most agent-to-agent communication follow predictable patterns, such as subtask delegation, result synthesis, conflict resolution. Hermes topology comprises a user request at the top, an orchestrator in the middle, and a message router and worker pools at the bottom.

The system separates agents into two tiers: Worker agents, which are stateless and focused on narrow subtasks, and Specialist agents, which are stateful and responsible for reasoning-heavy steps. Workers never communicate directly with each other; they only interact through the orchestrator, which enforces a directed acyclic communication graph.

This design prevents the combinatorial explosion of agent-to-agent messaging and provides an auditable execution plan.

Hermes introduces a minimal schema for agent messages, including core message types like delegation, result, conflict, and escalation. Each message carries an ID, type, sender and recipient identifiers, optional payload, context reference, and TTL. This structured communication protocol enables monitoring and debugging capabilities, as structured error propagation is more manageable than parsing free-form reasoning traces from an unstructured system.

The cost model in Hermes is cost-aware. It implements a budget-aware routing layer that estimates token costs before execution by analyzing the task decomposition. If the estimated cost exceeds a predefined budget, the router can compress the agent graph, merging redundant specialists or falling back to cheaper models for lower-priority steps.

This heuristic optimization prevents runaway costs on complex requests without imposing a hard limit.

On the other hand, LobeHub took a fundamentally different approach to multi-agent coordination. Instead of a rigid top-down orchestration, LobeHub treats agent coordination as a peer-to-peer mesh with gossip-style consensus. The LobeHub mesh comprises multiple agents communicating directly with each other, exchanging information and decisions through a decentralized network.

This approach allows for more flexible and dynamic communication patterns, as agents can form and dissolve connections based on the requirements of the task at hand. However, this design also introduces challenges related to consensus, fault tolerance, and ensuring the consistency of information across the network.

Both Hermes and LobeHub demonstrate that building scalable multi-agent systems requires addressing communication complexity, orchestration overhead, and cost management. Hermes's structured, top-down approach, with its typed interaction protocols and budget-aware routing, provides a clear audit trail and cost control mechanism. In contrast, LobeHub's decentralized, peer-to-peer mesh design emphasizes flexibility and dynamic communication patterns but requires careful consideration of consensus and fault tolerance mechanisms.

Understanding the architectural decisions and trade-offs of these two approaches offers valuable insights into the design space for building robust and scalable multi-agent systems.