Building the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil
Jerry Tworek and Rohan Anil, founders of Core Automation, argue that the transformer architecture has reached its limits and the primary bottleneck to smarter AI systems is now architectural. They contend that current models lack continual learning and test-time adaptation capabilities, critical for real-world deployment. They advocate for new architectures that learn from experience more efficiently than current reinforcement learning, optimize pre-training and RL end-to-end, and build a highly automated lab focused on accelerating innovation through kernel generation, aiming for models that can improve themselves without human intervention.
The discussion with Jerry Tworek and Rohan Anil, founders of Core Automation, centers on their contrarian premise that the transformer architecture has reached its limits, and the primary bottleneck to achieving smarter AI systems is no longer computational scale but architectural innovation. Jerry begins by acknowledging the immense contribution of transformers, stating that “the first step to replacing transformers is appreciating deeply how far they were able to carry us.” He emphasizes understanding their strengths—their economic viability, having mastered large-scale pre-training and reinforcement learning (RL), and enabling practical applications like Codex. However, he asserts that the architecture itself is now the bottleneck to developing more capable models. Current efforts often focus on making transformers cheaper or more efficient, rather than more powerful or expressive. Jerry, having led reasoning at OpenAI and previously an “RL maximalist,” shares his disillusionment that even extensive scaling of RL did not lead to AGI as he once predicted. The core issue, he identifies, is the discrepancy between models trained in controlled lab environments and their deployment in the messy real world. Benchmarks, he argues, often mirror training data, failing to capture real-world complexities. This leads to the conclusion that future models must learn at test time, adapting continually with users and real-world data. Transformers, however, exhibit fundamental limitations in this regard: in-context learning is data-efficient but severely limited in scope (e.g., Codex needing compaction after ~20 minutes), while fine-tuning suffers from catastrophic forgetting and low data efficiency, neither of which are easily solvable. Jerry posits that a new algorithm, expressed at the architectural layer, is needed to enable learning over much longer horizons. Rohan Anil, from his background in pre-training and optimization, echoes these architectural concerns. He notes that when the transformer was first developed, its significance wasn’t immediately apparent until large-scale applications like GPT-2 and GPT-3 demonstrated its benefits. Rohan views architecture as a method of “spending computation.” While transformers are efficient for pre-training, current inference patterns, particularly “chain of thought token generation” and auto-regressive decoding in RL, are highly inefficient. He calls solutions like speculative decoding “band-aids.” A critical flaw of transformers, he states, is their “poor computational depth.” Despite the “deep learning” moniker, most transformers are shallow (around 100 layers), and truly deep representations haven’t been widely achieved. Chain-of-thought is an attempt to increase computational depth at inference, but it incurs significant token cost. Both founders agree that while LSTMs could technically be scaled, transformers excelled due to their economic viability—the cost of training was lower than the revenue generated. Jerry explains that major labs are currently locked in a “coding-agent race,” making them averse to investing in alternative architectures that might only yield results years later. This competitive landscape, driven by short-term gains with existing transformer technology, creates a niche for Core Automation to pursue fundamental architectural innovation. Rohan emphasizes the symbiotic relationship between optimization and architecture. Stronger optimizers can enable the training of more complex and harder-to-optimize models, and crucially, “your optimization algorithm defines what architectures you will discover.” He advocates for an end-to-end optimization of both pre-training and RL, arguing that their naive combination is currently inefficient. He believes current RL approaches spend compute inefficiently due to sparse feedback, making it an “easy target to get orders of magnitude on top of.” On the topic of biological learning efficiency, Rohan is skeptical of surpassing it with current hardware, highlighting biological brains’ ability to build their own circuits and the inefficiency of current neural networks on fixed hardware. Core Automation’s mission is to be “the most automated lab in the world,” focusing on maximizing human agency rather than eliminating humans from the loop. Jerry defines AGI as “a model that can improve itself without human in the loop,” a task currently beyond frontier models due to the persistent need for human involvement in the “human-LLM hybrid.” Their strategic roadmap involves accelerating architectural innovation by automating kernel generation. Rohan cites a QR kernel competition where a highly skilled human achieved a 60x speedup over standard solutions, while current frontier AI models “are not even close to solving this problem.” Automating this process—reducing the time from architectural idea to high-TFLOPS GPU execution—is deemed critical. The ultimate goal is to find systems that exhibit “meaningful long-term adaptability” and learn at test time, with a playful metric being whether the lab’s models get better at the Core Automation scientists’ work each day.