Agent Gateway in 60 Seconds: Governed LLM Traffic with TrustGate

Agent Gateway in 60 Seconds: Governed LLM Traffic with TrustGate

Most development teams begin by making direct calls to AI providers like OpenAI or Anthropic using their respective SDKs. However, as projects grow and involve multiple applications, providers, and security requirements, managing access to these models becomes complex. An AI/Agent Gateway like TrustGate can simplify this process.

TrustGate is an open-source, security-focused Agent Gateway written in Go. It acts as a middleman between your applications and the upstream LLM APIs or tool servers. By configuring a single gateway, you can manage routing, policies, and observability without having to modify the client code of each application.

TrustGate provides a unified interface to LLM APIs and multi-modal computation (MCP) tool servers. By changing the base URL and a couple of headers, you can route traffic to different providers, handle authentication, and enforce policies. This eliminates the need to scatter API keys and rate limits across various services.

Key features of TrustGate include:

- Multiple planes: Admin (8080), Proxy (8081) for OpenAI-compatible LLM traffic with plugins, and MCP (8082) for aggregated Model Context Protocol endpoint.

- Policies: Rate limits, token rates, request sizes, semantic caching, and CORS can be enforced before traffic reaches the providers.

- One-binary deployment: TrustGate is a single binary that is easy to install and configure using Docker and Git.

- Quick setup: A 60-second bring-up process is provided with a simple shell script that clones the repository, sets up dependencies, and starts the gateway.

- Health checks: curl commands are provided to test the health of Admin, Proxy, and MCP endpoints.

- Demo scripts: The repository includes example scripts, such as ./examples/curl-first-request/first-request.sh, that demonstrate how to create a governed chat completion using TrustGate and the OpenAI SDK.

- Agent support: TrustGate's MCP plane (8082) allows for aggregation of registered MCP targets, enabling agents to access multiple tool servers without each maintaining a private mesh. The example setup for Cursor agents is available in examples/mcp-cursor/.

When should you use TrustGate? It is ideal when you need central policy enforcement (who, what, and how quickly), multi-provider routing with fallback capabilities, and MCP aggregation for agents. While LiteLLM is a good choice for a simple multi-provider proxy, Helicone is better suited for analytics-focused observability needs.

TrustGate's main advantage lies in its governance and security features, including centralized policy enforcement and MCP support. For more information, visit the documentation at docs.neuraltrust.ai or explore the TrustGate repository at github.com/NeuralTrust/TrustGate. Stars on GitHub help others discover this valuable tool.