Why You Need Rate Limiting for MCP Servers (And How to Do It)

Proper rate limiting for MCP Servers is essential to prevent potential Denial of Service (DOS) attacks, memory hogging, excessive API bills, and server/system overload. To implement rate limiting for MCP using agentgateway, you'll need a Kubernetes cluster, agentgateway installed, and a GitHub account.

Rate limiting helps control the number of requests an Agent can make to an MCP Server within a specified timeframe. This prevents the application from consuming excessive RAM, which could lead to a system crash due to lack of available resources. LLMs (Language Models) may retry when an error occurs, but this can result in high API bills or system crashes if not properly rate limited.

When implementing MCP rate limiting, the workflow involves a client (Agent) making HTTP POST requests to the MCP Server using JSON-RPC. The client first queries the tools/list endpoint to identify available MCP tools, and then caches the tool schemas for efficient communication. When an Agent requests an action, it uses cached tool information to perform the action.

To set up rate limiting for MCP in a Kubernetes environment, you'll create a Kubernetes Secret containing your GitHub Personal Access Token (PAT), configure a Gateway object, create an agentgatewaybackend to route requests to the GitHub Copilot MCP Server, and establish an HTTPRoute with the necessary route/path configuration. Finally, capture the Gateways IP and test the connection by making a POST request to the MCP Server.

With a configured Gateway and a rate limiting policy, you can prevent excessive requests to the MCP Server, ensuring stable system performance and preventing potential resource overload issues.