What happens now that AI is good at math? — the OpenAI Podcast Ep. 17
OpenAI researchers Sébastien Bubeck and Ernest Ryu discuss the dramatic and surprising progress of AI in mathematics. They cover how models went from basic arithmetic to solving Olympiad-level and even 40-year-old open research problems, what this progress means for the future of science and AGI, and the evolving role of human researchers in an era of AI-accelerated discovery.
The progress in AI’s mathematical capabilities over the last few years has been “nothing short of miraculous.” This leap has surprised even the experts, transforming models from being unable to solve simple scheduling problems to assisting Fields Medalists with their work and achieving gold-medal performance at the International Math Olympiad (IMO).
From Competition Math to a 42-Year-Old Open Problem
While IMO problems are challenging, they are “canned problems” with known, relatively short solutions. The true test is research-level mathematics. Researcher Ernest Ryu demonstrated this capability by using ChatGPT to solve a 42-year-old open problem in optimization theory concerning the Nesterov accelerated gradient method.
The process was a human-AI collaboration over 12 hours across three days. Ryu acted as the expert verifier and guide, correcting the model’s mistakes and steering the conversation toward novel approaches. The model, in turn, generated the proof, which Ryu and the model itself then verified. This marked one of the first instances of a genuinely open mathematical problem being solved with AI assistance, showcasing a move from solving known problems to discovering new mathematics.
Why Math is a Critical Benchmark for AGI
Mathematics serves as a perfect benchmark for AI progress for two primary reasons:
- Clarity and Verifiability: Mathematical problems are unambiguous, and their solutions are definitively correct or incorrect, making progress easy to measure.
- Long-Chain Reasoning: Solving advanced math requires sustained, consistent logical thinking. A single mistake in a chain of reasoning invalidates the entire argument. Training models on mathematics cultivates this robust reasoning capability, a skill that is expected to generalize to other scientific and complex domains. This is analogous to why humans are trained in mathematics—to develop rigorous, logical thinking.
Tackling the Erdős Problems: From Search to Discovery
A significant demonstration of AI’s power has been its work on the famous open problems posed by mathematician Paul Erdős. This progress occurred in two phases:
- Phase 1: Deep Literature Search: Initially, GPT demonstrated an advanced ability to conduct vast literature searches. It found solutions to some Erdős problems by identifying relevant theorems in unrelated mathematical fields, translating them from a different mathematical “language,” and connecting the disparate pieces of work.
- Phase 2: Novel Solutions: More recently, the models have gone beyond search to true invention. Researchers at OpenAI have used internal models to generate over 10 completely new, publishable solutions to Erdős problems that did not previously exist in any literature. This marks a significant milestone in AI’s ability to create original knowledge.
The Future Vision: The Automated Researcher and “AGI Time”
The current human-AI interaction is akin to a professor guiding a brilliant student. To tackle grand scientific challenges, however, a more autonomous system is needed. This leads to the concept of an automated researcher—an AI or collection of AIs that can work on a problem for extended periods.
This is framed by the concept of “AGI Time”: the duration an AI can think consistently. In four years, models have progressed from thinking for seconds, to minutes, to hours, and are now at the level of days or a week. The research goal is to extend this to weeks and months, enabling the AI to handle the long timelines required for major breakthroughs in fields like biology or materials science.
The Evolving Role of Humans and a Word of Caution
The goal of this research is not to replace scientists but to empower them. In this new paradigm, the human’s role shifts to providing direction, taste, and purpose. Humans will guide AI toward solving problems that matter to humanity, like curing diseases, while the AI accelerates the process of discovery.
However, this progress comes with a significant risk: developing a shallower understanding by over-relying on AI tools. Expertise is becoming more, not less, valuable. The ability to produce state-of-the-art results comes from a deep human understanding guiding the tool. Non-experts attempting to use these models to solve major theorems often produce incorrect proofs, highlighting the need for continued human expertise and rigorous training.
For those looking to learn, ChatGPT is an unparalleled educational tool. It can act as a personalized tutor, explaining complex topics like Maxwell’s equations, answering tailored questions, and making the solitary process of learning mathematics a more interactive and social endeavor.