Andrej Karpathy: From Vibe Coding to Agentic Engineering
Andrej Karpathy discusses the shift from 'vibe coding' to 'agentic engineering,' explaining why LLMs should be treated as 'ghosts'—jagged, statistical entities—rather than animals. He delves into the Software 3.0 paradigm, the limits of verifiability, and why human understanding remains the ultimate bottleneck in an age of outsourced thinking.
Andrej Karpathy explains his recent declaration of feeling “more behind as a programmer than ever,” a sentiment that crystallized around December 2023. This was a turning point where the latest generation of agentic AI tools became significantly more reliable. The code they generated “just came out fine,” fostering a new level of trust and marking a stark transition in capability that many who only experienced earlier versions of ChatGPT might have missed. This shift heralds a new era of computing.
The Dawn of Software 3.0
The paradigm of software development is undergoing a fundamental change, moving from explicit instructions to goal-oriented prompting.
- Software 1.0: Writing explicit code with programmatic rules.
- Software 2.0: Programming by curating datasets to train neural networks, where the “code” is the learned weights.
- Software 3.0: Prompting a Large Language Model (LLM). The context window becomes the primary lever to direct a powerful interpreter that performs computations in the digital information space.
A prime example of this shift is the installation process for openclaw. Instead of a complex, brittle shell script (Software 1.0) designed to handle countless edge cases across different environments, the new method is a block of text copied to an agent. The agent uses its own intelligence to understand the goal, inspect the local environment, and execute the necessary steps, debugging in real-time. This is a far more robust and powerful approach.
This new paradigm doesn’t just accelerate existing tasks; it makes entire categories of applications obsolete. Karpathy contrasts his MenuGen app—a traditional web application involving OCR, image generation, and a custom UI—with the Software 3.0 approach: simply giving a photo of the menu to a multimodal model and asking it to overlay generated images. The latter achieves the goal directly, proving the original app shouldn’t need to exist. This highlights a move towards new capabilities, such as creating LLM-powered wikis from unstructured documents, which were not possible before.
Vibe Coding vs. Agentic Engineering
While “vibe coding” captured the initial, more exploratory phase of working with LLMs, a more serious discipline is now emerging.
- Vibe Coding: Raises the floor for everyone, making software creation more accessible.
- Agentic Engineering: A professional discipline focused on preserving the high-quality bar of production software. It’s about how to coordinate powerful, yet stochastic and fallible, agents to build robust, secure systems at an accelerated pace.
This new discipline is creating a massive ceiling for developer productivity, far exceeding the old “10x engineer” concept. To identify these skills, hiring practices must evolve from solving puzzles to evaluating a candidate’s ability to build and secure a large-scale project using a suite of agentic tools.
Navigating Jagged Intelligence: LLMs as Ghosts
LLMs exhibit a “jagged” capability profile—they can be superhuman at one task and fail at a shockingly simple one. Karpathy gives the example of a model that can find a zero-day vulnerability yet advises walking to a car wash 50 meters away to wash a car. This jaggedness arises because models are trained via reinforcement learning on tasks that are easily verifiable. Their capabilities spike in domains like math and code, where correctness can be easily rewarded, and where the training labs have chosen to focus their data and compute.
To work with these models effectively, it’s better to think of them not as animals but as ghosts.
We’re not building animals, we are summoning ghosts. These are jagged forms of intelligence that are shaped by data and reward functions, but not by intrinsic motivation or fun or curiosity.
This framing encourages a more cautious and exploratory mindset. They are statistical entities, not conscious beings. You must probe their capabilities and understand that you are at the mercy of the “circuits” the labs decided to build. If your application falls outside these well-trodden paths, you will need to invest in your own fine-tuning.
The Enduring Value of Human Understanding
As agents automate more of the “how,” the human’s role shifts to providing taste, judgment, aesthetic sense, and high-level design oversight. Agents are like interns—they can handle the details (e.g., the difference between reshape and permute) but can make fundamental design errors, like attempting to link user accounts via email addresses instead of a persistent user ID.
You no longer need to memorize every API detail, but you must retain a deep understanding of the underlying concepts to direct the agents effectively. This leads to a crucial insight:
You can outsource your thinking, but you can’t outsource your understanding.
The human remains the bottleneck for understanding what to build and why it’s valuable. LLMs don’t possess true understanding; that remains the unique responsibility of the person directing them. Therefore, tools that enhance human understanding are becoming incredibly important. The future is an agent-native world, where infrastructure, documentation, and workflows are redesigned with agents as the primary user, decomposing the world into “sensors” and “actuators” that an AI can manipulate to achieve a goal.