How to Engineer a Multi-Agent Pipeline for Production Reliability

We constructed a pipeline composed of nine specialized agents that can transform a natural language request into functional code, a preview, and documentation in under four minutes. The system generates approximately 800 to 1,000 users a day, but its initial production iteration utilized around 30,000 tokens and made 15 to 20 calls to the high-tier model per request.

The main takeaway from this implementation of multi-agent AI is that increased agent quantity does not automatically solve larger problems. Instead, ensuring production reliability hinges on determining which tasks should be allocated to a model, establishing mandatory context, and replacing freeform generation with strict contracts.

Decomposing the workflow around failure boundaries is crucial. In our system, each agent is assigned a narrowly defined responsibility with explicit input and output. This separation enables individual testing, tuning, and model assignment based on the specific task.

However, dividing the workflow into nine agents is not without cost. Introducing additional boundaries adds orchestration, state transfer, and potential points of failure. The architecture was appropriate in our case because the workflow already contained distinct tasks with varying validation requirements. Only tasks that can be independently tested and validated without considering the entire result are suitable for their own stage.

Certain tasks, such as Schema Validation and Documentation Building, do not require large language model calls. By implementing these stages programmatically, we eliminated two potential sources of hallucination and reduced token consumption in those stages to zero. Separating the code generation, evaluation, and refinement processes also created a weak feedback loop when one agent generated code and then evaluated its own output.

Instead, we separated the evaluator from the refiner, with the evaluator applying strict rules and security scanning, and the refiner receiving specific findings to make targeted corrections.

Finally, the Model Context Protocol (MCP) provides access to live API documentation for the analysis, code generation, and evaluation agents through an indexed knowledge base. Context retrieval is capped to control context growth, and the MCP server operates as an isolated service to prevent competition for API quota with model calls.

Dynamic instruction injection is used to insert canonical examples for required APIs directly into downstream instructions, while optional retrieval is used for information where the model's knowledge may be lacking. This distinction allows for optional retrieval when the model may need information and direct injection of context when correctness depends on the model seeing the information.