How telemetry pipelines keep AI agent costs under control

As companies transition AI from experimentation to production, a new issue arises: increasing telemetry costs for AI agents. These agents, which are non-deterministic, iterative, and generate data at machine speed, are costly and difficult to monitor compared to traditional applications. A survey of over 300 enterprise IT decision-makers found that 59% have canceled or postponed AI deployments due to monitoring expenses.

Finance departments are often the ones to cut these programs, as AI projects often drain typical budgets. This phenomenon has been observed in major banks and other organizations. The situation is exacerbated by the exponential growth of telemetry volume, with AI/ML workloads driving a 54% increase in data in the past year. Companies are spending an average of $3.17 million on observability, with costs growing 28% year over year, and 83% consider AI observability a top priority.

The impending issue could be severe, with some organizations anticipating a 6X to 100X increase in telemetry data. This is due to the complex nature of agent tasks, which generate a multitude of traces, model calls, retrieval operations, tool calls, retries, and loops. Additionally, each agent, model, tool, and token creates high cardinality data, making aggregation expensive.

Early intervention is crucial to manage these costs effectively. By implementing a pipeline-first architecture that samples successful events and retains failures, retries, policy violations, and unusual traces, organizations can reduce redundant data and limit indexing costs. Millisecond-level context at the source allows for swift detection of issues before they escalate.