Stop counting tokens, start counting what ships

Developers utilizing AI agents for coding tasks don't merely receive an answer; they orchestrate complex workflows. A single task may require 10 to 50 times more computational tokens than a simple chat prompt. As AI agents become more proficient, the number of calls they make increases, potentially inflating expenses. This financial strain is already evident; Uber reportedly depleted its entire 2026 AI budget by April after intensifying Claude Code usage.

GitHub Copilot transitioned from fixed pricing to metered credits. Gartner projects AI coding token costs overtaking average developer salaries by 2028. The instinct is to impose usage caps, seat limits, and approvals before deploying agents. However, these blanket limits may stifle valuable workflows. Blanket limits merely optimize the wrong metric.

Cost per token gauges consumption, not outcomes. Certain expenditures reflect inefficiency like retries and uncontrolled loops, while others signify valuable software creation. A meter cannot differentiate between the two, nor can a financial cap. Measure the software that ships instead. A superior metric is cost per merged pull request: total AI-assisted development cost divided by the number of changes that pass review and are merged.

Include rework and defects in this calculation, so cheaper changes do not mask lower quality. If the cost decreases as AI usage rises—without sacrificing quality—developers are transforming consumption into valuable software. The focus should be on how much finished work the investment yields, not how many tokens are consumed. To refine this equation, two adjustments are necessary: reserve high-cost inference for tasks that require it, and reduce routine work costs.

Match the model to the task. Agentic coding does not necessitate the most powerful model for every step. Employ a frontier model or on-premises flagship for complex planning and inventive reasoning. Delegate suitable tasks, including code generation, test generation, and repeated refactoring, to open-weight models you manage. A gateway can direct requests based on task complexity, data sensitivity, cost, and latency.

This strategy reserves premium inference for circumstances that truly merit it, rather than imposing premium pricing on the entire workflow. Meter the exception, not the workload. Open-weight models make this distinction increasingly feasible. Benchmark studies indicate leading open-weight models can rival frontier models on certain tasks, though equivalence across all repositories and workflows remains unproven.

Evaluate candidates against your own code, tests, and acceptance criteria before assigning production work. Next, determine who owns the computational capacity. With rented inference, each call adds to the bill. With owned capacity, most costs are incurred upfront, making each additional call cheaper as usage expands. The critical factor is the fully loaded cost per merged pull request.

Owned capacity also provides a unique benefit invisible on invoices: control over sensitive code. By running code generation and testing on-premises, proprietary code remains within the enterprise. Only approved planning prompts and controlled exceptions are sent to external models, through a managed gateway the organization oversees.

This approach opens restricted repositories where security typically impedes adoption, and where significant benefits await. Test the operating model. At HPE, a hybrid AI platform routes over 150 internal applications' routine workloads to open-weight models on-premises, including on HPE Private Cloud AI. The outcome: token costs are reduced by 30 to 50%.

This exemplifies the benefits of hybrid models. With HPE Private Cloud AI, owning inference does not mean abandoning frontier models. Utilize volume on owned capacity while employing a gateway to handle complex tasks. Data also demonstrates that usage, not licensed headcount, drives expenditure. The top 5% of users generate most of the consumption, and the most intensive users consume roughly 450 times more than the least on the same license.

If the number of power users increases from a minority to half the workforce, monthly spend could grow approximately 4.4 times with constant headcount. HPE recommends tracking two metrics: cost per active developer and cost per merged pull request. Baseline both costs before altering infrastructure. Then run a full development cycle on owned capacity.

If both metrics improve as usage increases, your current model may still be preferable. Ownership is not for everyone. Low or unpredictable usage favors API-based solutions, where no capital is tied up in idle infrastructure. Ownership becomes worthwhile when sustained demand keeps it busy. Most enterprises will adopt a hybrid approach, utilizing frontier APIs for demanding or sporadic work, and owned capacity for repetitive tasks.

A controlled gateway directs what leaves the premises, enabling the benefits of frontier models without relinquishing control over your code. As developers increasingly rely on AI coding agents, usage will continue to rise. The key decision is whether this growth manifests as a higher variable-cost bill or as more software produced using capacity you own.

Companies leading AI-assisted development will not ration adoption to manage costs. They will construct an operating model where increased AI usage enhances economics rather than undermines them. To explore HPE Private Cloud AI further, visit HPE.