The Mean Looked Fine. Goodput Said No.
Mean latency is a liar. Goodput is the number I keep now. I learned this the boring way. I pointed a small agent loop at a free model endpoint. The average looked fine. Users still complained. Why?
- Mean latency can be misleading as it ignores slowest users
- Goodput measures share of requests meeting latency budget
- Redefining success as goodput reveals poor performance for some users