Agentic AI costs set to balloon fivefold by 2028

By 2028, the expense associated with agentic AI workflows is anticipated to surge more than fivefold as users embrace more intricate applications of the technology. Nvidia and other tech leaders are positioning inference and agentic AI as the subsequent phase of the AI wave, yet Gartner, an analyst firm, cautions that the cost of deploying these systems will escalate even as foundation models become more affordable.

The company has scrutinized the emerging realm of AI agents – autonomous systems striving towards a goal – and identifies multiple hurdles. Aside from substantial security concerns, these software agents are substantially more intricate than chatbots. Gartner contends that the decreasing model prices are enticing users to develop more elaborate workflows; the enhanced token usage may offset such savings, consequently driving up overall inference costs.

Innovation is progressing at a pace that outstrips the cost curve, according to Gartner. The financial intricacies of the inference enigma are epitomized by the disparities between a rudimentary chatbot and an AI agent, elucidates Gartner senior director analyst Will Sommer. While a simple chatbot must parse and interpret a query and swiftly provide a probabilistic reasonable response, an AI agent must continuously reason, negotiate, and question itself.

These processes culminate in higher inference costs – at least fivefold, and possibly more, for complex tasks. Ensuring a return on investment from such advanced AI tools necessitates either significantly greater returns than basic models offer or improved optimization of inference, routing, and orchestration, Gartner warns. This could involve assigning each task to the most cost-effective model capable of handling it.

The shift by some AI providers from flat-rate subscriptions to usage-based billing, as previously reported by The Register, has not alleviated the issue, as token-heavy workflows can trigger runaway costs under the new model. Gartner projected earlier this year that 40 percent of organizations would demote or decommission AI agents due to complications stemming from the highly publicized technology.

Moreover, the firm forecasted last year that at least half of all generative AI projects would surpass their budgets because of suboptimal architectural decisions and a lack of expertise, with most attempts to construct custom models likely to be abandoned.