{
  "id": 17957518,
  "title": "Tracking cost per finished job, not per token: the 3 fields I log",
  "url": "https://urgent.news/2026/11/03/tracking-cost-per-finished-job-not-per-token-the-3-fields-i-log",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-03T11:46:09.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/karmabox/tracking-cost-per-finished-job-not-per-token-the-3-fields-i-log-4842"
  },
  "original_language": "en",
  "account": "Most teams track the cost of their AI agents down to the individual token. However, few focus on the one crucial metric that determines the bill: whether the output was accepted. Before presenting this approach, it should be noted that KarmaBox, the company behind this schema, sells hardware for running agents locally, so there is a vested interest in emphasizing that tokens are not the sole factor. This schema is designed to be vendor-neutral and can be applied to any stack, whether cloud-based or local. It is important to verify these claims against your own data. The per-token metrics often mislead, as they fail to account for the actual costs involved. For instance, a standard task with 4,000 input tokens, 800 output tokens, 10% retries, and 70% cache hit can range between $0.00086 and $0.0603 depending on the model, resulting in a seventy-fold difference in price. A mid-range model typically costs around $0.0121 per task. This simplistic calculation can lead to a significant underestimation of real costs. There are two main issues with per-token metrics. Firstly, agents often loop, with each step re-reading context and invoking tool results, which are all billable inputs. Failures also trigger retries, causing the real bill to exceed the calculated amount by an order of magnitude. Secondly, and more critically, the cost of human time spent reviewing and reworking output is often overlooked. In one October cost analysis covering 1,840 accepted deliverables, model fees accounted for only 18% of the total cost, while human time spent on reviewing and correcting output made up 41%. The primary focus should be on optimizing the latter. The suggested minimal schema focuses on three key fields to measure the actual costs reaching your profit and loss statement. The first field, attempt_cost, represents everything that each attempt consumed, such as model fees or, in the case of running on your own hardware, an amortized share of the monthly hardware cost. It is advisable not to strive for precision in this field, as it will be the smallest number in the calculation. The challenge lies in capturing all billable events, as agents often overlook retries and tool-call round trips that inadvertently inflate costs. Implementing a structural approach, such as metering at a client wrapper that intercepts all model calls, can help ensure that every call is accounted for. The second field, review_minutes, tracks the time a human spends reviewing the output before making a decision on its fate. This metric is frequently overlooked due to its perceived inconvenience, yet it plays a crucial role in the overall cost. Capturing this data should be a natural byproduct of the reviewer's work, rather than a forced task that is abandoned. A practical method is to start the timer when the output is first opened, pause it during any idle time (such as not scrolling or typing for approximately 90 seconds), and stop it when the verdict is clicked. If there is no review UI, timestamps can be added to the verdict action itself. The third field, accepted, is a simple Boolean value indicating whether the output was successfully shipped or not. A redo is treated as a new row with its own attempt_cost and review_minutes, ensuring that rework is not hidden within the original cost calculation. The collection trap here involves the entity responsible for granting the accept/reject verdict. This responsibility typically lies with the individual accountable for the deliverable, be it a boss, customer, or auditor, rather than the engineer who built the agent or the operator who executed it. Consistently recording the verdict at the time of ship time, rather than in a delayed batch process, ensures that any missed or corrected outputs are accurately reflected in the cost calculation. Two common edge cases that arise when first implementing this schema are defining what constitutes a job and handling partial acceptance. It is best to use the unit a human would accept or reject as the job definition, such as a finished draft or reviewed document, rather than an LLM call. If a pipeline involves multiple agents producing a single deliverable, that should be recorded as a single job, with the attempt_cost encompassing the cost of all agents. Partial acceptance, where only part of the output is accepted, should be treated as rejected, with the salvage time logged as review_minutes on the redo row. Adhering to these strict rules will ensure that the metric remains accurate and meaningful.",
  "summary": "Most teams instrument their agents down to the token. Almost nobody instruments the one bit that decides the bill: whether the output was accepted. Quick disclosure before anything else: I work on KarmaBox, which sells hardware that runs agents locally — so I have an obvious interest in you believing that tokens aren't the whole story. The schema below is deliberately vendor-neutral. It works on…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}