#define AI Engineer - Greg Brockman, OpenAI (ft. Jensen Huang, NVIDIA)
Greg Brockman discusses his journey from a math enthusiast to a programmer, his early days scaling Stripe, and the core philosophies that drive OpenAI. He covers the critical partnership between research and engineering, the future of coding with agentic systems, and the immense infrastructure and algorithmic challenges on the path to AGI.
The Allure of Building: From Math to Code
My initial passion was for mathematics, inspired by figures like Galois and Gauss who worked on incredibly long time horizons. The thought was, “if anything that I come up with is ever used while I’m still alive, it wasn’t long-term enough.” This changed when I tried to publish a chemistry textbook I’d written. A friend suggested that instead of the difficult process of self-publishing, I should just build a website for it.
That led me to a PHP tutorial on W3 Schools. The first thing I built was a simple table-sorting widget. The moment I clicked the column header and the table sorted exactly as I had envisioned it—that was magic. It was a profound realization. With math, you write a proof, and maybe three people in the world will ever care. With programming, you write code—which maybe only three people will ever read—but everyone gets the benefit. The thing that was in your head becomes real and tangible in the world. I knew then that I just wanted to build.
Lessons from Scaling Stripe
The journey at Stripe, from a handful of people to over 250, was defined by an intense focus on solving problems from first principles. People often underestimate how hard the early days were. A key practice was adding all our customers on GChat, ensuring constant contact that was the next best thing to sitting over their shoulder.
A formative experience was our integration with Wells Fargo. We were told it would be a nine-month process, which was untenable for a startup. Instead, we treated it like a college problem set. In 24 hours, with John Collison testing from the top-down and I implementing, we had a working integration. During the certification call, we encountered an error. The certifier expected to reschedule for the next week, the typical cadence for their clients. But Patrick kept her on the line talking while I frantically debugged and redeployed the code live. We got five turns at it, failed, but she was kind enough to reschedule for two hours later, and we passed. We compressed six weeks of typical development into a single morning by refusing to accept the arbitrary constraints of how other organizations work.
This principle is more relevant than ever. By thinking from first principles, you can identify where processes are slow due to constraints that no longer apply to your specific situation, especially in a world where AI is dramatically accelerating productivity.
The Journey to AI and the AGI Conviction
My path to machine learning was one of self-study. After leaving Stripe, I started exploring AI, building a GPU rig and tackling Kaggle competitions. The real conviction in AGI, however, came from reading Alan Turing’s 1950 paper, “Computing Machinery and Intelligence.” What struck me wasn’t the famous Turing Test, but his proposed method for achieving it. He argued that you could never explicitly program all the necessary rules. Instead, he envisioned a “child machine” that learns like a human, through rewards and punishments. This concept felt fundamental: a machine that could understand and solve problems that you, the programmer, cannot fully grasp yourself.
Initially, I was disillusioned when my university NLP professor handed me parse trees, which felt far from Turing’s vision. But the rise of deep learning changed everything. Starting with AlexNet in 2012, we saw a general learning machine that, with enough compute, outperformed decades of specialized, rule-based research in computer vision. Then, it began to be the best in every other field—NLP, translation, and more. It was tearing down the walls between disciplines. This was the technology Turing had envisioned. It became clear that the entire 70-year history of computing was building towards the moment we could finally build these systems.
The Symbiosis of Research and Engineering
At OpenAI, we believe great engineers contribute at the same level as great researchers, and this is more true today than ever. The early days of deep learning may have been more about a researcher with a novel idea. Today, progress requires building systems at a massive scale—orchestrating 100,000 GPUs or designing complex Reinforcement Learning systems. The idea is critical, but without the engineering to make it a reality, it won’t see the light of day. The partnership between Ilia Sutskever’s vision and Alex Krizhevsky’s engineering on AlexNet is emblematic of this philosophy.
This relationship is something you never fully solve; you just advance to the next level of sophistication. Early on, we saw friction between engineering and research mindsets. An engineer sees an interface and assumes the implementation behind it doesn’t matter. A researcher knows that a subtle bug anywhere in the stack won’t throw an exception but will just silently degrade performance, so they feel responsible for understanding everything.
The key to bridging this is technical humility. When engineers join our world from a traditional software background, their intuitions don’t always apply. The most important thing is to listen, assume there’s something you’re missing, and deeply understand the why behind the current state before trying to change it.
The Future of Code and Development
Vibe Coding and Agentic Systems
The GPT-4 launch demo, where we created a website from a hand-drawn sketch, introduced the world to “vibe coding.” I see this as an empowerment mechanism and a representation of what’s to come. Today, it’s about making cool apps from scratch. Tomorrow, it will be about transforming existing applications—tackling the hard, un-fun work of migrating legacy codebases that companies are sitting on.
Ultimately, I believe the interactive loop of vibe coding will be overtaken by more agentic systems. You’ll have hundreds or thousands of agents working for you in the cloud, hooked up to various tools, making progress even while your laptop is closed.
How Codex Changes Coding
We’ve found that the way you structure your codebase directly impacts how much you can get out of models like Codex. Codebases have traditionally been optimized for the strengths of human cognition. To get the most out of AI, you must optimize for the strengths of models.
This means building smaller, well-documented, well-tested modules. The model can fill in the implementation details and run the tests a thousand times more than you ever would. This sounds a lot like good software engineering practice, but it’s now enforced by the capabilities of the tool. In a way, to get the most out of our most advanced models, we need to structure our code for a more “junior developer” to consume.
Scaling, Infrastructure, and Future Bottlenecks
As we scale, we face immense challenges in reliability and infrastructure. When training long-horizon agents that run for days, you can’t just restart a failed run. This necessitates sophisticated systems for checkpointing and recovering state, not just for the model but for the entire ecosystem of tools it interacts with.
Looking ahead, the infrastructure required is mind-boggling. There’s a fundamental tension between workloads that require massive, high-throughput compute for research and those that need extremely low-latency inference for real-time interaction. While one could design two types of specialized accelerators, predicting the right ratio is nearly impossible. In practice, the field adapts. When there’s an imbalance of resources, researchers invent new techniques to exploit it, like Mixture of Experts (MoE) models, which were a way to utilize an abundance of memory.
When considering the bottlenecks to building GPT-6 and beyond, it’s a balance of compute, data, algorithms, and money. But the most exciting development is that basic research is back. We’re at a scale where just throwing more compute and data at a vanilla Transformer is no longer the only answer. Algorithms are once again a primary driver of progress. The RL paradigm that made GPT-4 so much more reliable was a deliberate, multi-year research investment to give the model a form of “experience” beyond just reading text. This is the path forward: identifying the clear, missing capabilities and pushing on new paradigms until they work.