AI cloud operator Groq raises $350M more in funding

Artificial intelligence company Groq Inc. has secured an additional $350 million in funding, bringing its total capital raised to $650 million. The Series A round was spearheaded by returning investor Disruptive. Groq revealed that Nvidia Corp. is expected to join the funding round in the future, though the specific amount has not been disclosed.

The recent capital injection comes shortly after Groq concluded a $650 million funding round in March 2021. Established in 2016, Groq initially concentrated on creating AI accelerators. In December 2020, Nvidia signed a $20 billion agreement to license Groq's chip technology and bring several of its executives on board. Following this, Groq shifted its strategy and introduced a public cloud platform specifically designed for AI workloads.

This platform relies on the proprietary chip technology Groq developed before its partnership with Nvidia. Nvidia subsequently utilized Groq's technology to develop a chip called the Groq 3 LPU, which was unveiled in March and is optimized for inference, the stage where AI models are executed in production after being trained. The Groq 3 LPU functions in tandem with Nvidia's Rubin graphics processing unit.

Customers can run their models' feed-forward network (FFN) modules on the former chip while transmitting attention-related calculations to the GPUs. Nvidia asserts that this approach of partitioning processing tasks is more cost-effective than utilizing conventional graphics cards. The Groq 3 LPU also delivers performance enhancements in specific scenarios, such as when the model employs a mixture-of-experts architecture or incorporates speculative decoding, a technique that accelerates computations by delegating them to a secondary, more energy-efficient model.

Nvidia distributes the Groq 3 LPU within racks, each housing 256 accelerators. Groq leverages these racks to power its AI-optimized public cloud, known as GroqCloud. The cloud provides bare-metal environments that allow customers to tailor the underlying hardware. For users less familiar with technical aspects, Groq offers a toolkit called GroqStack to automate infrastructure management tasks.

Importantly, Groq's platform is not limited to inference; it also supports AI training. However, the Groq 3 LPU does not support training workloads. Groq intends to utilize the funds raised in this round to expand its public cloud. Currently, the platform operates on hardware housed in 13 data centers across the globe. Groq aims to increase its cloud capacity from 57 megawatts to over 200 megawatts within the following year.