Microservices vs Monolith: The Quest for the Holy Grail (Inspired by Monty Python)

The story begins with the narrator grappling with performance issues in a monolithic application. They compare this to the Monty Python scene where King Arthur insists on pushing forward despite obvious damage. The core problem is that the monolith's lack of scalability leads to performance degradation during traffic spikes.

The narrator then introduces the concept of a rate limiter as a potential solution. They question whether to implement the limiter within the monolith or as a separate microservice. This question ultimately boils down to the critical issue of state sharing across service boundaries.

In a monolith, the rate limiter can be implemented in-memory using a token bucket algorithm. However, when the system is split into microservices, in-memory state becomes unreliable due to each service instance maintaining its own copy of the state. Scaling out the service instances further exacerbates the problem, effectively multiplying the allowed throughput.

The "holy grail" of the article is found in centralizing the rate limiter state into a shared datastore, such as Redis. By moving the counter to Redis, the rate limiter becomes a thin service that all instances consult before allowing a request through. This approach ensures exact limits regardless of the number of instances running, offers low latency, and provides operational simplicity through a single place to tune, monitor, and debug.

The article provides an example of a naive in-memory limiter in Python, highlighting the issue of each service instance maintaining its own state. It then presents the solution using a Redis-backed token bucket implemented in Lua script, which guarantees atomicity and consistency across service instances. The Lua script allows a request if there are enough tokens remaining in the bucket, and updates the bucket's state accordingly.

The Redis connection is established, and the script is executed with the necessary arguments for capacity, refill rate, and current time.

The article concludes by emphasizing the benefits of the Redis-backed solution, including exact limits, low latency, and operational simplicity. It warns against common pitfalls, such as using separate Redis keys per instance or ad-hoc solutions like database tables, which introduce performance issues and defeat the purpose of a lightweight guardrail.