From metal to model: Private cloud gets an assembly line for production AI

As companies transition artificial intelligence from experimental phases to full production deployment, a significant hurdle emerges: the underlying infrastructure. Costs, tokenomics, data privacy concerns, and the manual labor involved in assembling GPUs, servers, networking components, and software stacks often hinder scaling AI workloads.

To address these challenges, Broadcom Inc., led by its Chief Marketing Officer Prashanth Shenoy, is leveraging turnkey automation to streamline infrastructure setup. Shenoy stated, "Setting up GPUs, servers, networking, Kubernetes, containers, AI software stack, testing, validating which models to use. It’s an extremely manual and complex process.”

He emphasized the need for this transition to occur within an organization's existing data center environment to avoid additional complexities around data privacy and model proximity.

Broadcom's solution, the VMware Cloud Foundation (VCF), aims to simplify the deployment of production AI workloads through automated hardware and software configurations. Shenoy explained that VCF integrates traditional workloads with AI components seamlessly, allowing organizations to avoid managing separate AI infrastructure. AMD's contribution to this ecosystem includes a range of GPU accelerators tailored to different model sizes—starting with the MI350P for smaller enterprises and progressing to the MI355X for the most extensive models.

Corporate Vice President of Software and Solutions at AMD, Raghu Nambiar, highlighted how AMD's hardware flexibility caters to diverse AI requirements, with specific accelerators designed for small, medium, and large models. The report includes a video interview with Shenoy and Nambiar, discussing these topics further during VMware Explore.