Tutorial: Show the Token Ceiling Before the Agent Runs Unattended
Last month a two-person team demoed an agent to a room of twelve people. At minute nine the run stopped dead. Half a draft hung on the projector. Nobody knew the token ceiling, so nobody could explain the stop. Who owned that decision? The PM did. She clicked "run twice" ninety minutes earlier. That click was the point of reversibility, and nobody marked it. So let me ask you something. Would you…
This tutorial aims to establish a reliable token ceiling for an agent before it runs unattended. A team demonstrated an agent to a group of twelve people, but the run stopped at minute nine due to a lack of understanding about the token ceiling. The two-person team, led by the PM, clicked the run twice ninety minutes earlier, marking the point of reversibility. However, this crucial information was not marked, leading to the need for this tutorial.
The tutorial begins by measuring one call before automating ten. It emphasizes measuring a token delta, latency, and exit code. A simple HTTP request using curl is provided to fetch the token usage from the response body, ensuring that the counter is based on actual provider data rather than a hypothesis. The tutorial stresses the importance of checking the total usage twice with the same fixture, ensuring that the ceiling math is safe.
Next, the allowance is converted into a per-run budget. A free allowance is a pool, not a promise, and it gets shared and spent. The tutorial proposes a design hypothesis of spending at most 12% of the measured allowance for a single run, leaving room for a retry, a bad day, and a second teammate. The ceiling function is defined to calculate the ceiling based on the allowance total, reserve floor, and allowance fraction.
The gate that refuses the run is then built. A custom exception, StopRun, is defined to raise when the allowance is insufficient or when a dry run is requested. The guarded_run function checks the allowance left against the budget and raises a StopRun exception if necessary. This exception must be handled properly, converting warnings into stop conditions rather than hopes.
To make the stop legible, the tutorial suggests rendering the stop as a persistent card with text, not color. The stop card should be accessible and provide a clear next step, such as reviewing the last complete draft or reviewing the previous draft. The focus should be moved to the stop heading, ensuring that screen reader users can hear the information.
Before the demo, the tutorial advises rehearsing the failure on purpose. Killing the run at 80% of its budget and asking a colleague to recover the draft tests the recovery story. If the colleague hesitates for more than ten seconds, the recovery story is considered fiction. The dry run and live run results should leave the same audit line, confirming that the evidence on the card approves the unattended run.
Finally, the tutorial presents an evidence-based decision-making process using a stop table. Evidence from the card, such as measured one-call tokens, agreement between two runs, a source for the ceiling, reserve floor intactness, focus-tested stop card, legible failure, and inferred tokens from character count, determine whether an unattended run is approved. If any evidence is missing, the decision is no.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
