Production AI shouldn’t need another stack. But can private cloud deliver?

Private AI is gaining traction in production, prompting enterprises to seek turnkey solutions rather than piecing together AI components. According to Prashanth Shenoy, chief marketing officer and vice president of Broadcom's Cloud Foundation Division, the challenge lies in turning this complexity into a streamlined system that enables faster AI deployment.

Shenoy argues that enterprises need a single, integrated platform to run their AI models and agents, rather than assembling a stack themselves. This single platform would address concerns around cost, tokenomics, security, and privacy of their data.

Shenoy emphasized that private cloud is the preferred platform for deploying production AI workloads, as it allows organizations to avoid creating silos and deploying AI workloads on the same infrastructure used for VMs and containers. Broadcom has achieved this by certifying servers from multiple vendors, working with AMD and Nvidia on the chip layer, and partnering with MetalSoft Cloud on heterogeneous firmware and hardware bring-up.

This integrated approach is delivered through the VCF Ops Console, simplifying operations and reducing provisioning time from months or weeks to minutes.

The selection of AI models and hardware certification is crucial, with Broadcom testing and optimizing about 150 models, ranging from open-source and open-weight models to commercial options. An AI gateway enables connection to over 40 cloud model providers. Shenoy noted that not every enterprise use case requires a frontier LLM model; rather, it's about purpose, fit, governance, and cost.

Security is another critical aspect, with a focus on frontier AI. Broadcom hardened VCF 9.1 with monthly patch releases and introduced new certifications to equip practitioners. The company's Frontier AI Security Readiness Program, covering assessment, architecture, implementation, and upskilling, aims to help organizations prepare for AI-driven threats that move at an accelerated pace.

Shenoy highlighted that the speed, volume, and variety of these threats have exploded, with attackers moving at AI speed, not weeks or months.

In summary, Broadcom is working towards a turnkey solution for private AI, integrating servers, chips, and software into a unified operational layer. This approach eliminates the need for customers to create parallel, siloed infrastructures and ensures the same unified operations management and security for AI workloads as for VMs and containers.