Why NVIDIA builds their own open models | Nemotron w/ Bryan Catanzaro

Bryan Catanzaro, NVIDIA's VP of Applied Deep Learning Research, explains the business rationale behind developing open models like Nemotron. The strategy is twofold: to drive internal systems R&D for future hardware and to support the broader AI ecosystem, which in turn expands NVIDIA's market.

February 4, 2026 4 min read

Why NVIDIA builds their own open models | Nemotron w/ Bryan Catanzaro — source: https://www.youtube.com/watch?v=Y3Vb6ecvfpU

NVIDIA’s investment in open-source models is a strategic business decision, not a charitable endeavor. The growth of AI directly correlates with the growth of NVIDIA’s market. Open models are crucial for developers and researchers, and while closed, API-based models are valuable, the open ecosystem requires more energy and support to ensure a healthy, diverse future for AI. NVIDIA views AI as a form of infrastructure, similar to the internet. Its development on open technologies allowed different sectors of the economy to adapt and integrate it in unique ways. Similarly, providing open, customizable AI models enables organizations to incorporate this technology into the core of their businesses more effectively.

The Two-Fold Mission of Nemotron

The Nemotron project serves two primary, interconnected purposes for NVIDIA:

Driving Systems R&D: The first and most critical job for Nemotron is to ensure NVIDIA’s future as a company. As an accelerated computing company, NVIDIA must prioritize which workloads to accelerate. To build the right hardware for the future of AI, NVIDIA must deeply understand the workloads, including nuances like numeric precision, network design, and sparsity. They cannot simply survey customers, as this information is often expensive to derive and closely held. By building state-of-the-art models internally, NVIDIA gains firsthand knowledge required to design its next generation of GPUs and systems. This makes the investment in Nemotron essential for the company’s core business.
Supporting the AI Ecosystem: The second job is to foster the broader AI community. NVIDIA holds a unique position, working with a vast range of companies, from startups to hyperscalers and global enterprises. It is in NVIDIA’s self-interest to support the entire ecosystem; the more people building, inventing, and deploying AI, the more opportunity there is for NVIDIA’s platform. This is a continuation of a long-standing strategy, dating back to the early days of CUDA, where outreach to academic and industrial communities led to breakthroughs like AlexNet, which ignited the deep learning era.

Beyond the Model: Datasets and Open Techniques

Nemotron is more than just a model; it’s a comprehensive effort to support openly developed AI. Because NVIDIA’s business is centered on systems rather than the models themselves, the company can afford to be more open with its learnings.

This includes:

Releasing high-quality datasets: In 2025, Nemotron began releasing not only post-training data but also pre-training data. This is a significant contribution, as high-quality, openly licensed pre-training data is scarce and provides fundamental research infrastructure for the entire community.
Sharing techniques and research: NVIDIA is open about its findings from expensive experiments, such as data blending recipes and optimizer settings. The goal is for others to build upon this work.

Scaling Collaboration: “Invitation, Not Control”

Coordinating the 500+ full-time contributors to Nemotron (with up to 2,000 people involved company-wide) required a shift in organizational strategy. The core principle is “invitation, not control.” Instead of top-down mandates, the leadership creates conditions where it makes sense for people to volunteer and collaborate.

Key elements of this structure include:

Decentralized Responsibility: The project is divided into roughly 20 areas, each with a designated “pilot in command.” This person’s job is not to form a committee but to gather the best ideas, facilitate data-driven decisions, and ultimately be responsible for “landing the airplane” in their domain.
Internal Openness: An internal website tracks all ideas and experiments. This transparency makes it clear how and where people can contribute, raising the overall rigor of the work.
Integration Studies: A focus on “integration studies” is crucial. In contrast to academic ablation studies that isolate a single variable, integration studies test how a hundred different ideas work together. This approach is essential for building a cohesive, large-scale system and fosters a collaborative team-first mindset.

The Evolution of Megatron

Megatron, the software framework Nemotron is trained on, has its own history.

It was created to demonstrate that NVIDIA systems were state-of-the-art for training large Transformer models, directly countering claims from competitors at the time.
Initially, the goal was raw performance and scale, not usability. Over the years, responding to community feedback, the team has invested heavily in improving software engineering to make it more accessible, though it remains a complex, powerful tool.
There is a persistent, healthy tension at NVIDIA between prioritizing hardware speed and developer speed. Improving the developer experience is crucial for allowing more people to unlock the hardware’s full potential.

Philosophy and Future Outlook

AI should be viewed as infrastructure that turbocharges companies, not as a singular event that will make humans obsolete. Every organization possesses unique data, domain knowledge, and customer understanding. Openly developed AI provides them with the tools to leverage these unique assets.

The future of AI will be driven by “intelligence per second.” The focus shouldn’t be on an abstract measure of intelligence but on how efficiently a model can be trained, fine-tuned, and deployed. Models that are optimized for the underlying hardware and incorporate compute acceleration will ultimately win because they can be iterated on more quickly and trained on more data.