TTFT is not TTFB: what 45 AI APIs measured from 4 regions actually show

Measuring AI API performance can be confusing due to two commonly used metrics: TTFB (time to first byte) and TTFT (time to first token). TTFB measures the time from sending a request to receiving the first byte of data, encompassing network latency, DNS resolution, TCP connect, and TLS handshake. TTFT, on the other hand, measures the time from sending a request until the first token appears in the model's response, accounting for network latency, queuing at the provider, and model prefill during token generation.

Two examples from São Paulo over the last 24 hours demonstrate this difference: openrouter API has a TTFB of 58 ms but a TTFT of 757 ms, while cerebras API has a TTFB of 184 ms but a TTFT of 1021 ms. The same region and probes yielded different winners for these metrics. The time to first token consists of network time (3%–33%) and provider queueing and model prefill (67%–97%).

This means that a benchmark focusing solely on TTFT might be measuring the model's performance rather than the API. The models powering these APIs vary greatly, with four different models listed behind the measurements, emphasizing that ranking providers based solely on TTFT would be misleading. Latency benchmarks should clarify whether they measure TTFB or TTFT, specify the model used, the region, the percentile (p50 or p95), how often the benchmark was run, and the number of samples collected.

This reporter only measures TTFT for a subset of the 45 tracked providers due to the need for paid API keys and the cost of token usage.