A Token Bucket Is More Than a Counter: 11 Tests for a Python Rate Limiter

A token bucket is more than just a simple counter. It comprises two independent values: Rate r, representing how many tokens are replenished per second, and Capacity C, which signifies the maximum burst allowed irrespective of the bucket's idle period. If T tokens are currently present in the bucket and Δt seconds have elapsed since its last update, the new balance can be calculated as T_new = min(C, T + max(0, Δt) × r).

The min(C, ...) ensures that an idle service does not accumulate unlimited request credit, preventing the burst limit from becoming irrelevant. For instance, with rate=2 and capacity=3, the first three cost-one requests are granted immediately; the fourth request is denied. After half a second, one token should become available again, even if a week passes; the bucket still contains at most three tokens.

Weighted requests are also supported, meaning the limiter can handle requests of varying costs, such as limiter.allow() costing 1 and limiter.allow(1.5) costing 1.5. In case of a high-cost request, the limiter should fail without consuming any tokens if the requested balance isn't available.