Nvidia building 1-trillion-parameter Nemotron 4 to rival open AI models: Report

Nvidia is constructing a groundbreaking AI model series called Nemotron 4, with the ambition of rivaling premier open-source models worldwide, according to The Information. The firm, a major player in the U.S., is one of the few to release open-source models, which have gained traction this year amid escalating AI costs and the emergence of cheaper Chinese alternatives nearing the prowess of top American labs like Anthropic and OpenAI.

The surge in recent cyber attacks involving autonomous AI agents has also heightened interest, particularly since open models lack safeguards for cybersecurity applications.

The largest Nemotron 4 model is anticipated to contain a minimum of 1 trillion parameters, as stated by numerous employees working on the project. Nvidia has not fixed a release date for Nemotron 4 and has yet to finalize the extensive training necessary. Nevertheless, employees predict the model could be operational by late fall, per The Information's report. Nvidia has cited prior statements indicating its involvement in Nemotron 4 development but has not verified the specifics reported by The Information.

Kari Briski, Nvidia's vice president of generative AI, commented in an email that the company invests in Nemotron 4 because it believes every company and country requires accessible frontier open models to bolster safety and security, spur innovation, and provide a reliable foundation for future generations. Nvidia has formed an alliance with other companies last month to develop and distribute AI safety and cybersecurity tools.

Additionally, the firm endorsed an open letter co-signed by tech powerhouses such as Microsoft, advocating for open-weight models to ensure innovation doesn't migrate overseas.

On the same day as The Information's report, Nvidia unveiled Nemotron 3.5 Lightning, an enhancement to its lineup geared towards code review, tool utilization, security alert monitoring, billing inquiry answering, and other duties. Nvidia also unveiled NeMo Switchyard, an open-source model-routing library, enabling automatic allocation of AI tasks to the most appropriate models.