Anthropic Co-founder: Building Claude Code, Lessons From GPT-3 & LLM System Design
Tom Brown, co-founder of Anthropic, shares his journey from a YC founder to a key figure behind AI's scaling breakthroughs. He discusses the discovery of scaling laws that underpinned GPT-3, the mission-driven founding of Anthropic, the surprising success of Claude for coding, and his perspective on what he calls "humanity's largest infrastructure buildout ever."
Tom Brown’s journey into AI was unconventional. After early startup experiences at YC companies like Linked Language and Grouper, he developed a “wolf” mentality—the need to proactively hunt for solutions rather than waiting for tasks. This mindset proved crucial for his later work. Despite feeling academically underprepared (famously earning a “B-minus in linear algebra”), he decided to pivot to AI research in 2014. He crafted a six-month self-study curriculum focused on machine learning fundamentals, including Coursera courses, Kaggle projects, and foundational textbooks, to build the skills necessary to contribute to the field.
OpenAI and the GPT-3 Inflection Point
Joining OpenAI, Brown initially worked on engineering infrastructure, such as building a Starcraft environment, rather than direct ML research. His career, and the industry, hit an inflection point with the discovery of scaling laws.
The original scaling laws paper just the like very straight line over like 12 orders of magnitude. I’m just like 12 orders of magnitude is… a stupidly large amount… that convinced me to definitely pivot all of my work into scaling.
This empirical observation—that model performance predictably improves with more compute, data, and parameters—provided a clear, albeit controversial, path forward. At the time, many researchers viewed this “brute force” approach as inelegant, but its effectiveness was undeniable. Brown was instrumental in building the infrastructure for GPT-3, which involved a critical shift from TPUs to GPUs. This move was driven less by the hardware itself and more by the superior software stack (PyTorch over TensorFlow), which enabled faster iteration and experimentation.
The Genesis of Anthropic
The core group that founded Anthropic spun out of OpenAI with a shared, intense focus on AI safety. They took the implications of the scaling laws seriously, believing that the eventual handoff from human to AI control required an institution built from the ground up to handle that responsibility.
Starting during the COVID-19 pandemic, the initial team of seven co-founders had no concrete product plan but a powerful, unifying mission. This mission-driven culture was key to attracting the first hundred employees and has helped the organization scale to over 2,000 people while minimizing internal politics.
The Rise of Claude and the “User-as-Model” Insight
Anthropic’s first product was an internal Slackbot version of Claude. The team hesitated to launch it publicly, uncertain about its potential impact and lacking the necessary serving infrastructure—a hesitation Brown now sees as a learning experience. The launch of ChatGPT served as a “wake-up call,” prompting Anthropic to release its API.
The company’s major breakthrough came with Claude 3.5 Sonnet, whose success and rapid market adoption, especially for coding, surprised even the internal team. Brown attributes Claude’s superior coding performance to a different evaluation philosophy.
- Avoiding “Teaching to the Test”: Anthropic does not have a dedicated team focused on optimizing performance on public benchmarks. Brown believes this practice creates bad incentives.
- Focus on Internal Value: Instead, the team focuses on internal benchmarks and extensive dogfooding, with a primary goal of accelerating their own engineers’ workflows.
The development of Claude Code revealed a powerful insight. The project began as an internal tool built by an engineer for other engineers. Its success stemmed from a crucial mind-shift: treating Claude itself as the user.
“I think the users is Claude. It’s like give Claude the right tools that Claude can actually do that effectively… This team was the most focused on Claude as like a user which I think makes sense that you guys would understand Claude the best.”
This approach of building tools to empower the model, rather than just the human end-user, is a rich area for innovation that startups can explore.
Humanity’s Largest Infrastructure Buildout
Brown oversees Anthropic’s compute infrastructure and describes the current AI expansion as “humanity’s largest infrastructure buildout of all time.” He notes that spending on AGI compute is on a trajectory to grow 3x per year, and the primary bottleneck for this expansion is power, particularly in the United States.
To navigate this landscape, Anthropic employs a multi-chip strategy, utilizing GPUs, TPUs, and AWS Trainium. While this approach increases the complexity of performance engineering, it offers significant advantages:
- Flexibility: Access to a larger total pool of available compute.
- Optimization: The ability to use the right chip for the right job, matching hardware strengths to specific tasks like training or inference.
Advice for the Next Generation
For young engineers looking to enter the AI field, Brown advises them to focus on intrinsic motivation over extrinsic credentials. His core message is to take more risks and pursue work that a more idealized version of themselves would be proud of, rather than chasing conventional career paths.