The Enterprise Fight Against Runaway AI Costs

In the pursuit of controlling the escalating costs of AI adoption, enterprises are employing three strategic measures: agent harness, model selection, and inference economics. Rather than solely focusing on model capabilities, the effectiveness of AI systems is now evaluated based on their ability to generate measurable productivity gains without causing soaring project expenses.

As AI transitions from pilot projects to everyday enterprise workflows, the costs associated with each model call, token, inference, and computing resource accumulate, necessitating a reevaluation of the entire AI system's design.

The first strategy involves the agent harness, a software layer responsible for managing how an AI agent utilizes a model. It determines what information the model receives, how a task is divided, and how frequently the model is invoked. An example of this is Sarvam AI's Sarvam Code, an agent that distributes engineering tasks among planner, worker, and verifier agents.

By routing work between its Sarvam 105B model and a more powerful open model, Sarvam achieved an average cost of $2 per task, compared to $4.1 to $27.8 for other models such as Claude Code and OpenAI's Codex. This system reduced serving costs by approximately 40%. The agent harness minimizes unnecessary model calls by preventing the model from performing redundant work, caching previously processed information, and ensuring only the necessary data is sent to the model, resulting in significant cost reductions.

The second strategy is model selection, which involves employing more affordable models for routine tasks while reserving expensive, powerful models for complex work. By routing simple requests to smaller models and reserving frontier models for intricate assignments, enterprises can lower the cost per AI-powered transaction without sacrificing outcome quality.

Krupesh Bhat, founder and CEO of Melento, emphasizes the importance of matching model costs to the difficulty of tasks. By monitoring the number of requests handled by smaller models, the average cost per transaction, and the task success rate, companies can assess whether this approach leads to cost savings without compromising outcomes.

This strategy illustrates the distinction between productivity gains and actual cost reductions, as increased efficiency must translate into lower spending on models, personnel, or rework to be considered truly cost-effective.

The third strategy focuses on inference economics, which aims to minimize the cost of running the remaining AI after unnecessary calls and model selection have been optimized. Arjun Nagulapally, CTO at AIONOS, explains that substantial savings can be achieved by removing AI from tasks that do not require it, such as customer account lookups, eligibility checks, or field validations.

In a telecom project processing over a million interactions monthly, AIONOS redirected around 30% of routine steps to conventional software and utilized an open-weight model trained on resolved customer queries. By reusing information and shifting some work to more cost-effective computing resources, AIONOS reduced the total cost of resolving each interaction by 35%, with the majority of savings stemming from tasks that no longer necessitated AI intervention.

The combined effects of agent harness, model selection, and inference economics offer enterprises a comprehensive approach to managing AI adoption costs effectively.