Not Memory. Not Energy. This Is the Biggest Bottleneck in Artificial Intelligence, and These Are the Companies That Can Benefit.

Artificial intelligence data centers face several bottlenecks, but capital constraints could prove the most significant. Building new data centers requires securing a site, ensuring it can supply adequate power and water, constructing the facility, and finally installing racks with AI chips and networking equipment. Securing enough power to run each data center is a challenge, while wafer shortages have affected the semiconductor industry.

Memory chips, in particular, are in short supply, as memory chipmakers hesitated to expand production capacity, fearing another downturn.

However, the biggest bottleneck may be capital. Long lead times and massive price tags for data centers mean capital constraints could hinder some companies from building as rapidly as desired. The biggest hyperscalers, such as Alphabet (GOOG), Amazon (AMZN), Meta Platforms, Oracle, and Microsoft, are experiencing negative free cash flow due to increased AI spending.

To fund their future build-outs, these companies have sought financing in the debt and equity markets. Neoclouds and leading AI labs face even more significant capital challenges, as they lack alternative revenue streams and the creditworthiness to support the substantial loans required for continued expansion.

Some companies, like Nvidia and Broadcom, have chosen to secure financing for their customers to ensure they can keep purchasing hardware. This approach creates loyal customers and helps resolve the financing bottleneck, but it also introduces additional risks. The most effective way to overcome the financing challenge is to reduce the cost of building new data centers and installing hardware.

The primary expense is the chips themselves, with a clear shift toward custom silicon, or XPUs, for a growing share of computing. XPUs offer lower upfront costs and improved price-performance for specific tasks.

Custom chips, or XPUs, can save cloud platforms considerable capital expenses over time. During Amazon's first-quarter earnings call, CEO Andy Jassy explained the potential savings from Amazon's custom Trainium chips, which could save tens of billions of dollars in capital expenditures annually. Similarly, Google Cloud's head, Thomas Kurian, reported that the payback period for custom TPU servers is half that of their average server.

As a result, XPU makers, led by market leader Broadcom, are expected to benefit significantly as companies seek to manage capital spending. Smaller competitors like Marvell Technology (MRVL) could also experience rapid growth as XPU sales surpass those of GPUs and other chips.