The Mean Looked Fine. Goodput Said No.
Mean latency is a liar. Goodput is the number I keep now. I learned this the boring way. I pointed a small agent loop at a free model endpoint. The average looked fine. Users still complained. Why? Because averages hide the tail. And free tiers live in the tail. Two free things, one real question The setup I wanted to test was simple. A free model endpoint, plus a free server to run the client…
The mean latency of a service can be misleading, as it ignores the slowest users. Goodput measures the share of requests that meet a latency budget, providing a more accurate picture of the user experience. The author tested this by sending 100 requests to a free model endpoint and measuring the time it took for each request to complete.
The average latency looked fine, but a few users were experiencing significant delays. The author then redefined success as goodput, the percentage of requests that completed within a specified latency budget. This revealed that even if the average response time was acceptable, a small fraction of users were still experiencing poor performance.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
