Token Buckets Do Not Belong on Free Inference
Admission control is a fence, not a conversation. A free language-model call is a variable-latency guess with an untrusted prompt surface, and it does not belong on the path that decides whether a request may enter a production service. The temptation is obvious in an industry that now asks models to classify almost everything. Abuse often looks like language. Support tickets look like language.…
We haven't written up this one. Dev.to has the full story — the link below goes straight to it.
