Rafay Systems targets the operating layer of the AI infrastructure boom

The AI infrastructure market is undergoing a crucial transformation. Initially focused on acquiring graphics processing units (GPUs) and establishing capacity, the next phase involves converting that expensive hardware into a secure, dependable, and lucrative cloud service. Rafay Systems Inc., co-founded and led by Haseeb Budhani, identifies this as its opportunity.

During a recent conversation, Budhani discussed the operational challenges faced by neoclouds, sovereign cloud providers, telecommunications firms, and enterprises as they deploy increasingly large AI systems. The demand for these systems is immense, with providers purchasing hardware at a massive scale and customers eagerly awaiting capacity.

However, mere GPU acquisition does not constitute a cloud. Operators still require orchestration, networking, security, multitenancy, observability, auditing, and a developer experience that enables customers to utilize infrastructure without extensive manual processes. Critical to this is minimizing the time to revenue. AI infrastructure is costly, depreciates rapidly, and must start generating returns as soon as possible.

Budhani emphasized that "the thing that matters most is faster time to market and a better user experience. And if you can deliver both of those things, everybody wins." Defining a cloud simply, Budhani stated that customers should be able to access an AI service via a portal or application programming interface (API) and receive it in a multitenant environment with the press of a button.

Today's AI cloud providers, lacking the extensive resources of industry giants like Amazon Web Services, Microsoft, and Google, cannot afford to construct every layer independently. Rafay's role is to offer the operating software bridging the hardware and customer experience. This software allows providers to offer bare metal, Kubernetes environments, virtual machines, serverless services, open-source models, and token-based offerings through a unified platform.

By addressing various customer segments—such as large model developers, bare metal users, serverless enthusiasts, and enterprises requiring serverless experiences or token purchases—Rafay enables AI clouds to cater to multiple needs. This breadth enhances revenue potential, with bare-metal capacity generating predictable earnings, while managed services and token consumption yield better margins.

The rise of sovereign AI is also reshaping the cloud landscape, as countries and regions seek local infrastructure to keep data and computing resources closer to home. "The big driver is sovereignty of compute, sovereignty of data," Budhani explained. Initially, this wave comprised local model builders and agentic application developers, but now, enterprise customers are following suit, bringing their existing expectations from hyperscale platforms—like quotas, policies, auditability, attestation, and security controls—along with the desire for a simple consumption experience.

Rafay's most significant investment has been its capability to expedite service deployment for customers. Rapid deployment, rather than just the provision of software, is now a crucial investment for the company. Budhani highlighted that the company actively engages with customers to identify their requirements, prepare the operating environment, and deploy services within days, not quarters.

The industry's deployment cycles, which once took extended periods, are now compressing into weeks or days, making swift turnaround essential for success in this rapidly evolving landscape. In practice, no single vendor can provide the complete AI stack. Deployments often involve a mix of technologies from various suppliers, including Nvidia GPUs, Dell Technologies servers, multiple networking providers, specialized storage, security platforms, and systems integrators.

Supply constraints add complexity to this heterogeneous environment, making the software layer adaptable to the available resources.