Everyone should be building with AI, and everyone should know its cost

Generic AI tools promise to solve every problem, from managing customer relationships to analyzing simple tasks like monitoring a cat's water intake. When used without defined roles, these tools fail to deliver results and erode trust in technology. As employees attempt to use these tools for every task, tokenmaxxing emerges - a pattern of repeatedly refining AI output until something usable comes out.

Some organizations blame employees for tokenmaxxing or double down on training on 'responsible use of AI'. However, traditional procurement systems were not designed for the rapid pace of AI software creation. AI development tools, like public LLM chatbots, can be opened and used with just a few clicks, while traditional IT procurement often takes months.

When employees use unvetted shadow AI tools for daily tasks, they bypass approval processes and incur costs that are hard to track. This can lead to poor code quality, lack of accessibility, and poor authentication practices. Restricting AI use for employees often results in personal costs, as they pay for subscriptions and use the tools anyway.

The pressure is on for organizations to show ROI on AI initiatives within a short timeframe, but many current AI projects are not delivering on this promise. Token consumption is expected to increase dramatically, while semiconductor providers are already experiencing significant cost reductions per token. However, the users of these tokens are evolving - organizations are building AI agents that act autonomously.

These agents can consume far more tokens than human prompting, especially when running unattended on unscoped problems. Many tokens go into prototypes that never see production, leading to wasted resources and inflated costs. When teams build with generic LLMs and no guardrails, senior engineers spend hours cleaning up the output, deepening the trust gap in technical teams.

Developers are frustrated by almost-right AI answers, and debugging AI-generated code can be time-consuming. The speed of AI code generation is not the issue; the problem lies in democratizing access without accountability and proper cost tracking. To address this, organizations should retire the all-purpose chat box and encourage focused AI use tied to specific business needs.

Users should be able to explain the purpose behind their token spend in simple terms. This new approach should promote accountability and ensure AI tools are used for genuine business needs, rather than becoming another source of frustration.