No Priors Ep. 123 | With ReflectionAI Co-Founder and CEO Misha Laskin
Misha Laskin, co-founder of Reflection AI and former researcher at Google DeepMind, discusses the company's mission to build superhuman autonomous systems. He introduces Asimov, a code comprehension agent designed to solve the 80% of an engineer's time spent on understanding complex systems, rather than just code generation. Laskin delves into the intricacies of co-designing product and research, the critical role of customer-driven evaluations, the bottlenecks in scaling reinforcement learning (RL) — particularly the "reward problem" — and why he believes the future is one of "jagged superintelligence" emerging in specific, high-value domains like coding.
The Mission: From Superintelligence to Autonomous Systems
The goal of building “super intelligent autonomous systems” is a more product-focused evolution of building general superintelligence. One approach is the industrial lab model, decoupled from product, which aims to conquer a broad set of benchmarks. This has led to successes in narrow domains, like AlphaGo.
The alternative approach, which Reflection AI is pursuing, is to co-design research and product. It involves working backward from how a superintelligent system would be deployed and used in the real world. This requires intense focus on a specific problem category, but ensures the system is optimized for real-world problems. The key is to pick a category that is “ASI-complete”—large and complex enough to necessitate and draw out superintelligent capabilities. Coding is one such category.
“It makes a lot more sense to be focused and co-design those two things together, the product and the research.”
Asimov: A Code Comprehension Agent
The majority of existing AI coding tools focus on code generation, which addresses only about 20% of a software engineer’s workload. The real bottleneck, consuming 80% of their time, is comprehending complex systems and collaborating with teammates to understand codebases that are too large for any single person to hold in their head.
Asimov is a code comprehension agent designed to tackle this core problem. It acts as an “omniscient oracle” for an engineering organization, functioning like a principal-level engineer with a deep, holistic understanding of the company’s systems.
It excels at “semantic queries”—vague, high-level questions that are difficult for other systems to handle because the necessary context is scattered across code, documentation, project management tools, and the team’s tribal knowledge.
An example of a query where Asimov shines: “My jobs are running five times more slowly than usual. Why is that?” Answering this requires synthesizing information from recent, seemingly unrelated pull requests from different teams that created a race condition.
Technical Approach: Co-design and Evaluation
Building a system like Asimov requires a tight coupling of product and research, allowing for innovation across the entire stack, from product features to agent design and model training.
- Product & Data: Asimov integrates not just with the codebase but also with other knowledge sources like chat logs and project management tools. It also features a “team-wide memory,” allowing engineers to teach the system organizational knowledge, which can be edited and versioned, similar to a Git workflow for meta-knowledge.
- Research & Models: The focus is on post-training models for specific capabilities driven by customer needs. This includes:
- Long-context reasoning: Using smaller models to process vast codebases and extract relevant information, akin to neural retrieval.
- Tool use and multi-hop reasoning: Fine-tuning agentic models on a specific set of tools (e.g., searching Jira, calling other models) to achieve depth in reasoning, rather than just general capability.
- Evaluation: The most critical component is evaluation. Instead of relying on academic benchmarks that may have a weak correlation to user needs, Reflection AI develops its evaluations directly from customer problems.
“Develop your evals based on what customers are saying and what they’re doing… you kind of work backwards and reverse engineer from what a user is asking for to what capabilities you want to drive in your system.”
The Bottleneck: A Reward-Bound World
The primary obstacle to scaling Reinforcement Learning (RL) for language models isn’t a lack of compute for training, but a structural one: the reward problem.
“I think it’s fundamentally a… we’re in a reward-bound world.”
To effectively train an agent, you need a reward model that can accurately describe the outcome of any arbitrary task. However, creating such a reward model is an “ASI-complete” problem in itself. By the time you have a neural network that can perfectly verify any outcome, you have likely already built a superintelligence.
Current RL methods are also inefficient at exploration and credit assignment. They can’t discern which specific step in a long reasoning chain was correct, leading to meandering, inefficient thought processes unlike structured human thinking.
The Startup Advantage and the Future of AI
A focused startup has an advantage over large, incumbent labs by tightly coupling product and research. While a large lab might have the best general-purpose model, it doesn’t guarantee the best product due to the organizational distance between research teams and end-users.
The future of AI will not be a single, monolithic AGI but a “jagged superintelligence,” with distinct, powerful systems emerging in specific, economically vital categories like coding, finance, or life sciences. The blueprint for creating these systems will be established, but deploying them will be a multi-decade endeavor, as each category requires its own specific research, product, and data solutions.
“There’s no such thing as generalization. There’s just bring the test distribution into train.”
This verticalization is already happening. Companies that build products in critical-path categories like coding but do not own their own intelligence will find themselves in an existential position as frontier labs build their own vertically integrated solutions.