Nvidia Reportedly Developing Massive Open-Source AI Model

Nvidia is currently developing "Nemotron-4," a massive open-source AI model featuring over one trillion parameters, according to tech-focused media outlet The Information on the 11th (local time).

The upcoming model is roughly twice the size of its current largest model, "Nemotron-3 Ultra," released last June, and aims to achieve performance on par with the world's leading open-source AI models.

On the same day, Nvidia also separately unveiled "Nemotron-3.5 Lightning," a lightweight model featuring 30 billion parameters created using a "distillation" technique from its larger models.

However, Nemotron-4 still falls short of Chinese models in both scale and performance.

It has significantly fewer parameters than China's latest open-source models, such as Moonshot's Kimi K3 (2.8 trillion), Alibaba's Qwen3.8-Max (2.4 trillion), and DeepSeek's V4-Pro (1.6 trillion).

Nemotron-3 Ultra also ranked second among U.S. open-source models in relevant benchmarks, showing a wide gap with China's top-tier models and falling outside the global top 40.

Nvidia's strategy is to offset this gap with compression technology that boosts the performance of smaller models.

Concerns that U.S. open-source models are lagging behind China have sparked regulatory debates within U.S. political circles regarding open-source models. Meanwhile, Nvidia CEO Jensen Huang has strongly defended open source, stating that open models "advance safety, cybersecurity, scientific progress, and national security."

While pre-training data and the architecture for Nemotron-4 have been partially finalized, its exact specifications and release date remain undecided, and final training has not yet concluded. It is reportedly expected to be completed as early as late this coming autumn.

Much more personnel is being poured into the development of this model than before.

While the research paper covering Nvidia's previous large-scale model involved 570 authors, employees stated that an even larger number of personnel will participate in Nemotron-4.

In line with this open-source drive, Nvidia has also substantially increased its spending on computing resources used for training its own models.

By re-leasing AI servers from cloud providers that purchase its chips, the company expanded its multi-year cloud service commitments to 28 billion dollars (approx. 394 trillion won) through early 2031, as of last April.

This amount is roughly triple compared to a year ago.


However, this strategy places Nvidia in a delicate position.

This is because it could effectively create a competitive dynamic with top-tier AI developers that are its largest customers, such as OpenAI, as well as open-source startups backed by Nvidia investments like Reflection AI and Thinking Machines Lab.

Nvidia has already invested 30 billion dollars (approx. 422조 won) in OpenAI alone.

Even so, calculations within and outside Nvidia suggest that as the variety of open-source models grows, the demand for its own chips will ultimately increase alongside them.

Anastasios Angelopoulos, CEO of AI model evaluation firm LMSYS Arena, stated, "No matter which of these companies builds great open source, Nvidia wins."

Nvidia receives training data and ideas through the "Nemotron Coalition," which includes Reflection AI, Cursor, Thinking Machines Lab, Mistral, Prime Intellect, and Cognition.

Prime Intellect provided 300,000 simulation environments to assist with model training.

Cognition is reportedly discussing the provision of coding data.

Naver Cloud also joined the coalition last June, though its specific role in the development of Nemotron-4 has not yet been disclosed.

Still, even Nvidia faces limits on the funds it can allocate to model development.

Its fiscal-year cloud service commitments stand at 7 billion dollars, which falls short of the spending by OpenAI and Anthropic.

However, this is higher than the cumulative funding raised so far by the majority of open-source model developers across both the United States and China.

(Photo: Getty Images, Yonhap News)
※ Please note: This article was translated by AI and may contain errors.