Rack-scale AI: How Dell and AMD are building infrastructure

Dell Technologies and AMD are collaborating to create rack-scale AI infrastructure that can keep up with evolving workloads and customer demands. The duo is focusing on enhancing server consolidation and modular solutions to simplify AI deployment and scaling. Dell's PowerEdge servers, powered by AMD processors, are designed to deliver 70% performance gains and enable the folding of 13 servers from Dell's 14th PowerEdge generation into one.

Dell's three-rack-unit server is more energy-efficient than a two-unit equivalent, allowing for higher rack density.

The partnership aims to create a modular solution, the Dell AI Platform with AMD, which integrates AMD Instinct accelerators and EPYC processors with Dell PowerEdge servers, networking and storage. This platform enables businesses to start with configurations that meet current workloads and expand compute and GPU density as needs grow. The on-premises enterprise AI solution supports training, fine-tuning and serving large language models using AMD ROCm software.

Dell recently announced support for AMD Instinct MI350P PCIe GPUs in its PowerEdge XE7745 and R7725 servers, offering enterprises an alternative for scaling agentic and generative AI deployments. The MI350P features 144 gigabytes of HBM3E memory, which can be a crucial factor in time-to-first-token latency. AMD and Dell's collaboration also includes PCIe integration, offering air-cooled deployments for users to deploy AI within existing data center infrastructure, which may help address the environmental impact and water-based cooling concerns.