Groq Raises $350 Million to Fund AI Inference Goals

Artificial intelligence (AI) inference specialist Groq has secured $350 million in fresh funding to bolster its capabilities. The Series A round, announced on Monday (Aug. 17), will help Groq cater to customers utilizing medium and large-sized Nvidia accelerated computing clusters for both training and inference purposes. Led by investment firm Disruptive, the round also saw Nvidia's participation, following a $20 billion licensing agreement between the two entities earlier this year.

Groq's executive chairman and CEO of Disruptive, Alex Davis, expressed enthusiasm about the partnership, emphasizing the critical role inference will play in AI infrastructure. Davis highlighted the company's expertise in scaling LPUs and delivering the performance, efficiency, and reliability required by the next generation of AI.

Groq operates 13 data centers globally, serving over six million developers, Fortune 500 enterprises, and numerous "AI-native companies." In June, the company raised $650 million. Earlier this year, Nvidia acquired technology from Groq and employed several team members, although Groq maintains its independence. Inference involves a trained AI model processing new data and generating results, a crucial stage for AI applications such as customer service chatbots or financial document analysis.

As companies deploy AI systems handling thousands or millions of requests daily, inference poses significant operational challenges and cost drivers. A PYMNTS report last fall underscored the increasing importance of inference over training for most enterprises, as inference takes place each time a user interacts with an AI system.