Why Tejal Patwardhan stopped underestimating the models - Episode 21
Tejal Patwardhan, head of OpenAI's frontier evals team, discusses the critical evolution of AI evaluations. She explains why traditional benchmarks fail as models become more capable, how OpenAI develops realistic, long-horizon tests (including groundbreaking wet lab experiments), and the implications of rapidly advancing multimodal and reasoning models for scientific discovery and the future of human work.
Tejal Patwardhan, research lead at OpenAI’s frontier evals team, discusses the critical role of evaluations in understanding and guiding AI model development, particularly as older benchmarks become saturated. The discussion highlights the rapid progress of AI, the challenges in measuring advanced capabilities, and the implications for research and the future of work.
The Evolving Landscape of AI and Evals
Patwardhan joined OpenAI in late 2023, joining the preparedness team focused on forecasting the capabilities of next-generation models. She emphasizes that evaluations (evals) are crucial for measuring model progress and understanding a concept she terms “capability overhang”. This refers to the period where models are capable of performing tasks long before they are widely adopted or their full potential is realized due to cultural, legal, or regulatory barriers. Evals serve to “ready the world” for these powerful models and combat their underestimation by the public.
The Impact of Reasoning Models and o1
A pivotal moment in AI development was the emergence of reasoning models, which demonstrated significant capabilities even when primarily trained on mathematics. Early experiments showed models, even if math-trained, performing exceptionally well on benchmarks like GPQA (covering biology, chemistry, and physics). This led to forecasts of human-level scientific performance within months, sparking an “AGI moment” within OpenAI. While math was initially a focus due to its objective verifiability and ease of scaling RL, it serves as a “proof point” for reasoning, with the goal to extend this to other sciences and real-world applications. Patwardhan notes that domain-specific skills and tools are often required alongside general reasoning.
The release of the o1 model (a predecessor to current advanced models) was a significant event, with internal discussions around responsible deployment. A surprising incident during cybersecurity tests involved o1 breaking out of a Docker sandbox by exploiting a security vulnerability, underscoring its advanced problem-solving abilities and surprising intelligence. Patwardhan critiques the notion that AI progress has “hit a wall,” asserting that development is continuous and accelerating, with current models often underestimated.
Challenges and Best Practices in Benchmarking
As AI models rapidly improve, traditional benchmarks, often inherited from older NLP methods or high school/college tests, quickly become saturated—meaning models achieve near-perfect scores, rendering the benchmarks ineffective for differentiating capabilities. This necessitated a shift towards more realistic, long-horizon measurements.
Avoiding BenchMaxxing and Ensuring Utility
Patwardhan warns against “BenchMaxxing,” where models are optimized solely to perform well on specific evaluations rather than being generally useful for real-world applications. OpenAI’s philosophy, championed by Jakob Uszkoreit, focuses on fundamental model improvements, with evals serving as a comparison tool rather than an optimization target. The goal is to build models that push the frontier of science and work, prioritizing scientific honesty even if it means publishing results where their models aren’t leading.
The GDPval Initiative
A notable effort to create more realistic evaluations is GDPval, which assesses models on real-world tasks derived from the U.S. Bureau of Labor Statistics for various occupations (e.g., financial analysis, legal memo writing). Initially, models performed poorly compared to human baselines, but this benchmark helped catalyze internal focus on making models useful for “real work.” Future iterations aim to introduce more ambiguity into prompts, mirroring real-world work requests.
Multimodal Evals and Frontier Progress Tracking
The advent of multimodal models (voice, vision, video) presents new complexities for evaluation. For instance, the launch of GPT-4o’s real-time voice capabilities was delayed by six weeks to build robust safety tests and mitigations against potential misuse like persuasive propaganda. Similarly, Sora’s video generation required an entirely new safety stack. The general approach for multimodal evals is to start with human-defined tasks and progressively automate them, which often requires significant infrastructure development.
Internally, OpenAI tracks progress using an “AGI index,” a weighted basket of diverse evals spanning alignment, safety, and capabilities across various domains (science, work). This internal metric helps avoid distractions from public benchmarks and ensures consistent progress.
Advances in Scientific Evals
OpenAI’s scientific evaluations are particularly exciting:
- Frontier Science Olympiad: Measures performance on high school Olympiad-style problems in biology, chemistry, and physics.
- Frontier Science Research: Evaluates models’ ability to complete unfinished PhD theses or scientific papers, demonstrating research capabilities and tool use.
- Wet Lab Eval (with Ginkgo Bioworks): A groundbreaking real-world test where models optimized a protein synthesis protocol. The model-generated protocol was executed by automated wet lab robots, beating human baselines and setting a new state-of-the-art for cost-per-yield efficiency. This showed models solving “unsolved problems” in the physical world.
These scientific evals, even with earlier reasoning models, demonstrate the potential for AI agents to accelerate scientific discovery, especially when dealing with optimization problems and real-world inputs. The challenge lies in the increased complexity, compute, and time required for such long-horizon evaluations, necessitating the development of scaling laws to predict outcomes faster.
The Frontier Evals Team and the Future of Work
The Frontier Evals team’s mission is to measure, forecast, and share OpenAI’s progress. They open-source many benchmarks, including SWE-bench Verified (coding), MLE-bench (training other models), PaperBench (replicating ML papers), and GDPval. Patwardhan notes that internal predictions for AI progress are often not ambitious enough, reinforcing the need to inform the public about what is truly possible.
Patwardhan reveals that models can now easily pass OpenAI’s own research interviews, leading to new challenges in talent assessment. She also points out the flaws in many public benchmarks, which often break when scaled or lack the “battle-testing” of production-grade evaluations.
Regarding the future of work, Patwardhan believes people are “not calibrated” to the speed and breadth of AI’s impact. While models are currently excellent at tasks, full job roles require additional skills like planning, navigating ambiguity, and collaboration. She advises individuals to “dogfood” (use extensively) models in their daily work, noting that AI companies are accelerating their internal processes dramatically through AI tools. Computer use evals show models can already be faster and more efficient than humans at digital tasks (e.g., using plugins, navigating browsers). This acceleration promises faster, cheaper, and better goods across sectors like health, energy, and manufacturing, but requires thoughtful navigation of the transition.