Agentic AI infrastructure shifts enterprise focus from model choice to platform control

As agentic AI infrastructure transitions from experimental stages to production, organizations are grappling with new challenges beyond selecting the appropriate model. The primary concerns now revolve around managing costs, data exposure, and the underlying infrastructure that support these production AI applications. Joe Fernandes, vice president and general manager of Red Hat's Artificial Intelligence Business Unit, emphasizes that organizations must reconsider relying solely on public cloud AI services, particularly as agentic systems evolve from basic assistants into persistent enterprise applications capable of acting across various business systems.

According to Fernandes, the escalating costs associated with AI models are becoming a significant issue, especially as companies scale up from pilot projects to full-fledged production systems. Initially, the focus was mainly on the cost of models, but now the data side of things is equally important. Companies must consider whether they are comfortable with their data being used in public cloud services and whether they have compliance or sovereign requirements that preclude such usage.

These combined factors are prompting enterprises to explore alternative solutions rather than relying exclusively on public cloud services.

Fernandes further underscores the growing importance of platform teams in managing AI systems. The platform function is increasingly central to AI strategy as agents become a new class of enterprise applications. While traditional reliability, scalability, and security remain crucial, autonomy introduces additional layers of operational responsibility.

Platform teams must ensure that these autonomous systems run reliably, securely, and at scale, addressing the same concerns previously focused on in conventional applications but now tailored to the unique demands of autonomous agents.

This complexity intensifies as enterprises shift from a few AI assistants to hundreds or thousands of agents operating across multiple systems. The hidden operational costs associated with managing such a vast number of agents, beyond just compute costs, include controlling access points within the network and ensuring that the actions of these agents are traceable.

To tackle these challenges, Red Hat is extending its open-source approach to encompass not just model access but also infrastructure and sandboxing mechanisms essential for running agents. One notable project in this space is Nvidia's OpenShell, an open-source sandbox runtime for AI agents that Red Hat actively contributes to and maintains.

OpenShell plays a pivotal role in addressing the need for control over agent behavior, including defining what systems and data the agents can access. By establishing "agent sandboxes," Red Hat is providing a framework to manage these complex operational aspects. Fernandes highlights that the same need for control extends to sovereignty and hybrid deployment strategies.

Enterprises are increasingly seeking control over where their models, agents, and data are processed, whether on public cloud services, private environments, sovereign clouds, or edge devices. This necessitates a hybrid world approach, ensuring that agentic AI infrastructure can span across various deployment environments while adhering to sovereignty requirements, a concept Red Hat has been advocating for years.