Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix)

What Actually Improves AI Applications

Many teams building AI products get distracted by the wrong things. The common belief is that success comes from staying updated with the latest AI news, adopting the newest agentic frameworks, agonizing over vector databases, or constantly evaluating which model is “smarter”.

However, the most significant improvements to AI applications come from fundamental product-building principles:

• Talking to users to understand their needs and feedback.
• Building reliable platforms and optimizing end-to-end workflows.
• Preparing better data, which is often the biggest lever for performance.
• Writing better prompts to guide the model effectively.

Instead of debating which new, untested technology to adopt, it’s more productive to ask: How much of an improvement will this actually provide? And how difficult will it be to switch out later? Over-committing to a new, unproven technology can be a significant risk.

Understanding Model Training: Pre-training vs. Post-training

The process of creating a powerful language model can be broken down into two main phases: pre-training and post-training.

Pre-training: Encoding Statistical Knowledge

Pre-training is the initial phase where a model learns the statistical patterns of language. It’s trained on a massive corpus of data (like a large portion of the internet) to predict the next “token” (a unit of text that can be a word or part of a word).

“I think of language modeling as a way of encoding statistical information about a language… when you train a large amount of data like it you see a lot of languages a lot of domains so it can tell like okay you basically say this sentence then if user do the prompt then it would come like with the next most likely token.”

This process builds a general capability and understanding of language, but pre-trained models are often raw and not directly usable for specific applications. As users and application developers, we almost never interact with a purely pre-trained model.

Post-training: Shaping Model Behavior

Post-training is where the magic happens and where most frontier labs now spend their energy. It takes the general capabilities of a pre-trained model and refines its behavior to be helpful, harmless, and aligned with human intent. The two primary methods are:

1. Supervised Fine-Tuning (SFT): The model is trained on a high-quality dataset of prompt-and-response pairs created by human experts. This teaches the model to emulate how an expert would respond to a given instruction. Many open-source models use a technique called “distillation,” where they are trained on the outputs of a more powerful model (like GPT-4) to emulate its behavior.

2. Reinforcement Learning (RL): This is a powerful technique for further refining the model’s behavior based on feedback.

The Crucial Role of Reinforcement Learning (RLHF)

Reinforcement Learning from Human Feedback (RLHF) is a key process in post-training. The goal is to “reinforce” or encourage the model to produce better outputs.

The process works as follows:

1. Generate Responses: The model produces two or more responses to a single prompt.
2. Gather Feedback: A human evaluator compares the responses and indicates which one is better. This comparative feedback is much more reliable and easier for humans to provide than an absolute score.
3. Train a Reward Model: This human preference data is used to train a separate “reward model.” The reward model’s job is to predict which response a human would prefer, essentially learning to score the quality of the main model’s outputs.
4. Fine-tune with RL: The main language model is then fine-tuned using reinforcement learning, with the reward model providing the signal to guide it toward generating higher-scoring (i.e., human-preferred) responses.

This feedback doesn’t have to come from humans. It can also be AI feedback (using a powerful model to judge a weaker one) or verifiable rewards (e.g., giving the model a math problem where the solution is known, providing a clear right/wrong signal).

The Pragmatic Approach to Evals

Evaluations (“evals”) are essential for understanding and improving model performance, but their application should be pragmatic. There’s a debate about their necessity, but the decision often comes down to return on investment.

• When Evals are Critical: For large-scale applications, core product features that are a competitive advantage, or in situations where failure has catastrophic consequences, rigorous evals are non-negotiable.
• When to Be Pragmatic: For a new, non-critical feature, if it’s “good enough” and users are happy, dedicating two engineers for several weeks to improve performance from 80% to 85% might not be the best use of resources compared to launching an entirely new feature.

Evals are not just about a final score; they are a tool for discovery. By analyzing where a product performs poorly, teams can uncover opportunities for significant improvement, which might be as simple as improving messaging to a specific user segment.

RAG: It’s All About the Data Preparation

Retrieval-Augmented Generation (RAG) is a technique that provides a model with relevant context from an external knowledge source to answer a question.

“The biggest performance in their RAG solutions coming from like better data preparations not agonizing over like what vector databases to use.”

While teams often fixate on choosing the right vector database, the most substantial gains in RAG performance come from how the data is prepared. Key data preparation techniques include:

• Chunking Strategy: Finding the right size for text chunks is a balance. Chunks must be small enough to be retrieved with precision but large enough to contain meaningful context.
• Adding Contextual Metadata: Augmenting chunks with summaries or hypothetical questions that the chunk can answer helps the retrieval system find the most relevant information.
• Data Rewriting: Reformatting source documents into a question-and-answer format can significantly improve retrieval accuracy.
• AI-Friendly Documentation: A lot of documentation is written for humans who have implicit context. It often needs to be annotated or rewritten to provide that context explicitly for an AI model.

AI Adoption and Productivity in the Enterprise

Measuring productivity gains from internal AI tools, like coding assistants, is notoriously difficult. This leads to interesting organizational dynamics.

One anecdote highlights a key challenge: when asked to choose between giving their team expensive coding assistant subscriptions or getting one additional headcount, line managers almost always choose the headcount. In contrast, VPs or executives often choose the AI tools. This is because the manager’s incentive is often tied to team size, whereas the executive is focused on broader business metrics and overall productivity.

The “Three-Bucket” Test

A company conducted an experiment by giving a coding assistant (Cursor) to half of its engineers, who were informally bucketed into high, average, and low performers. The results were telling:

• High performers saw the largest productivity gains, as they were already proactive and knew how to leverage the tool to solve problems more effectively.
• Average performers saw a moderate boost.
• Low performers saw the least benefit, sometimes using the tool to generate poor code on autopilot.

However, this isn’t universal. Other companies report that senior, high-performing engineers are the most resistant to AI tools because they have high standards and are more critical of the AI-generated code.

The Future of Engineering and System Thinking

AI is changing the structure of engineering teams. Some companies are shifting senior engineers away from hands-on coding and more toward high-level tasks like PR reviews, setting engineering standards, and defining processes. The code itself is increasingly produced by junior engineers or AI, which raises the question of how junior engineers will gain the experience needed to become senior.

This highlights the growing importance of system thinking. Computer science isn’t just about coding; it’s about understanding how complex systems work together to solve a problem. AI tools are good at well-defined, local tasks but struggle with debugging issues that span multiple components of a large system because they lack a holistic view. The most valuable skill for engineers of the future will be this ability to reason about systems at a high level.

• Tags:
• RAG
• Fine-Tuning
• Feature
• RLHF
• Evals
• AI Engineering
• Productivity
• Model Training
• System Thinking