Your Timer Is a Wall Clock, Not a Stopwatch: Four Elapsed-Time Myths and a Monotonic Probe

How long did that batch job take? Simple question. Your logs probably answer it wrong. Two numbers for one run appeared in the job log and dashboard. Neither was lying. They were reading different clocks. The following FAQ will debunk four common claims about elapsed time and provide a ten-minute probe to test the hypothesis.

The setup is straightforward: a single job calling a model once per input and generating a JSONL report.

Myth 1: time.time() minus time.time() is a duration. Everyone does this, but it's wrong. Wall clock represents what time it is, not how long a task took. Subtraction of two wall readings can yield negative numbers or include time not spent. Linux users can check NTP sync, adjustment, chrony offset and slew rate, and seconds since boot using timedatectl, chronyc tracking, and /proc/uptime. The monotonic clock, which never goes backward, should be used for measuring durations.

Myth 2: If your timer jumped, the host slept. This may be true, but it's not the only possibility. Gaps in monotonic time indicate the process did not run; it doesn't reveal the cause. Possible reasons include throttling, preemption, host suspension, or I/O blockage. To identify the issue, log both wall and monotonic clocks side by side and compare the differences.

Myth 3: A timeout means the request failed. It doesn't. A timeout is a local observation about your side of the socket and provides no information about the remote side. The remote work might have completed after you gave up. Blindly retrying can lead to duplicate results, writes, or charges. To avoid this, derive a dedupe key from the request, not the clock. Log the key before and after the call, and persist it to disk for identifying reruns versus new units of work.

Myth 4: Retries are free because the model is free. This is incorrect. Free is a price, not a property of your runtime. Every retry consumes wall time, which is your actual budget. Three retries can consume significant time, impacting your overall performance.